Mercury Grand Marquis possible Failing Alternator

Hey all, I have a couple of real straight forward questions about symptoms of a possibly failing (or failed) alternator on my vehicle. I have a 1995 Mercury Grand Marquis LS and this morning I went out to start the car and let it warm up a bit (I live in Minnesota and mornings are rather cold in winter). The vehicle cold started perfectly and the voltage gauge was reading normal, about smack dab in the middle between High and low. After about a 10 minute warm up I came back out to the vehicle to discover the voltage meter had dropped to just above the low indication, and the battery light was illuminated on the dashboard. I drove around hoping the alternator would charge the battery a bit but the voltage gauge continued to hover right around the very bottom. Looking back, I was starting to see some symptoms of a possibly failing alternator, when I was at stop lights my headlights would dim, if I had a lot of electronics on, the voltage meter would dip quite a bit. The battery is less than two years old and I’ve never had issues with it, even cold starting in these cold Minnesota winter mornings. I’ve also looked to make sure there were no loose connections in the wiring. I just wanted to ask the professionals if replacing the alternator would be a prudent plan.