This system is pretty basic, and thank goodness we can finally troubleshoot something that doesn't have all kinds of unnecessary, unreliable computers. Anything I can think of off the top of my head is going to involve draining the battery overnight and wires melted together but I wouldn't necessarily blame the voltage regulator for that.
How do you stop the engine now? With the engine stopped and the ignition switch off, measure the voltage at the ignition coil positive terminal. It should be 0 volts. There has to be voltage getting there for the engine to stay running. If you find voltage, follow that wire back. We have to determine where that voltage is coming from.