2004 Ford Mustang No Start

I have a 2004 Mustang GT, 34000 miles, and have recently had problems with it starting. I will get in the car and turn the ignition and nothing. However if I pump the clutch with the ignitions still turned, it will crank after a few pumps. This does not always work however. I thought it was the Nuetral saftey switch located at the top of the clutch, however, when I replaced the switch it did not fix the problem. My next thought would be the starter relay or maybe even the starter going bad. I might want to add that I recently modified the exhaust and there is no heat sheild on the starter. Might be the cause but not sure?