1994 Chevy Astro Heater temp on the fritz

Your temperature drops in cab because at a stoplight, your engine is idling, and not the waterpump is not pumping coolant with any real power at that time. Assuming that your idle speed is not to low, and your belts are fine and adjusted correctly,

The first thing you should check is that your coolant level is correct. This should be done when the engine is cool, (safety). If it is low, add the correct amount of coolant, you can get premixed almost everywhere now.

Also, your temperature gauge, (may indicate a lower temperature ) this might mean that your thermostat is not allowing the engine to warm up enough. Easy fix.

If your coolant and thermostat are fine, you may need a new waterpump, keep an I on the temp gauge, does it move up and down with changes in engine rpm? If you suspect this to be the case, take it to a trusted auto service shop and have them take a look at it. All of these parts are easily changed and not expensive. Failure of the coolant system will be.