At random times, car will not start when key is.

Based on your observation of "fiddling", it sounds like a problem inside the neutral safety switch. I never advocate replacing parts randomly without properly diagnosing the cause first, but in this case trying a new one could be a cheap solution. There's multiple designs though. If you want to tackle it yourself you'll have to list the engine size and transmission type. Even then there's multiple choices. Some have 7 wires, some have 2. Some are on the transmission. Some are on the steering column.