Weak alternator and making winding noise

Okay, been hearing a winding noise coming from my vehicles alternator. Other then the crazy winding noise, the truck drives good. But lately I notice the battery gauge in my truck flickering from 13.49 volts to 12.10 volts while the truck is idling or coming to a stop. But as soon as I give it a little gas the battery gauge goes back up around 13.49 volts. Now, I took the alternator to local parts store to be tested it tested good. Okay, my multi meter tested 13.49 volts running in park. When I add the lights, air conditioning, and radio, the volts drop down around 12 volts. So I think it's time to replace the alternator, but I was wondering would changing the brushes would take care of my charging problems? Is there a way to check the brushes to see if the need replacing? Napa auto parts have a brush replacement for $26.00 or alternator replacement for $225.00 to $400.00. Can you please help me out? Thank you guys.