1999 Pontiac Sunfire Engine stubmles

Hi, I have a 1999 Pontiac Sunfire 2.2L 4 door sedan that bought about a year ago. The problem was already there when I bought the car but I bought it anyway because I was in a hurry and was basically the only thing I could afford. The problem is that the car stumbles when I try to go at the same speed, if I push the gas more than needed to keep the same speed or leave my foot from the gas the problem goes away. For some reason the problem only presented about one or two times every 3 or 4 weeks for a few minutes or when I got into the highway for a while (20 mins or more), so I didn't paid too much attention (I didn't have too much to put into repairs either). During the spring and summer the problem increased and was there almost all the time, which started to worried me. Right now is happening one or two days a week. This is why I did during the summer that didn't eliminate the problem: change transmission oil, change spark plugs, change wire set for the spark plugs and change air filter. When I bought it I changed the gas filter. I passed the inspection a few weeks ago and the guy showed me that the exhaust was rusted and broken, he told me I needed to change the entire thing, including the catalytic converter. I was able to see that it was broken between the pipes and the engine.

I believe that catalytic converters have temp sensors. Changing it is going to help to address my problem (since it happens more during the summer than the winter the sensors work properly under low temperatures)?

also, when I bought the car the lady left some bills from her shop, I could appreciate that it had some sort of water pump problem. Could that have caused the problem I am having now? Also, the problem also happened when I drive on 1st gear. Could that eliminate the option of a transmission failure since there weren’t any gear changes involved?

Also, about one time a month the engine doesn’t want to start, but it does if I turn the key and push the gas at the same time... Could that indicate something?

Somebody told me to replace the fuel pump. Is there any (cheap) way to check if it’s working? Could that have something to do?

I am changing the exhaust system in a week.

thanks.