Taurs wont crank

2002 ford taurs 80000 miles wont crank
out of the blue got in and turned key and nothing
happened. Dash lights, interior lights all work untill key turned to start position. All lights then go dim, and nothing. No clicking or other sounds. When key is released power does not quickly or completely return.
checked starter fuse (40amp) under hood and removed and reinstalled starter relay, both no help. I can get the lights and power to return quickly if I pull off the 3 connectors at the bottom of the fuse box for the interior fuse box and put them back on. However the same thing happens again when I turn the key to start. I have moved the shifter full travel from P-1 and back with no change. Is the starter relay toast?How to check the relay itself and how to check the neutral switch? All lights are bright and good charge on battery and cabels are tight, clean and in good condition. Occasionally a clicking sound can be heard from the interior fuse box area, but I cant feel it when I touch the relays. Is there a relay in the interior fuse box which can cause the car not to crank?
any help is appreciated.
thanks