2002 Saturn L300 Electrical issues

I am having issues with my veh. When I drive or idling no matter where I go, all the lights in and outside of the car flicker. Example if I have my headlights on and roll down my window all the lights will go dim for just a second then go back to the regular brightness. When I accelerate the lights to not get brighter which I think may not be the alternator. The battery has been replaced in the veh so it is not a factory battery. I also noticed that when the car is not running and I am listening to the stereo and I turn it up the stereo cuts out. I think this is from not enough power is being sent to be used. I can not find out what to do with this issue. Any input would really help. Thanks.