Charging system malfunction

When I start my car and turn on some of my acessories (lights, radio etc) the car will start to lose power. When I turn everything off it will keep running. But when I keep it running for a few minutes with all acessories off the entire dashboard stops working ( speedometer, rpm gauge, temp gauge etc including my running lights, signal lights) then when I turn my headlights on and off the dashboard fickers. But the strangest part is, is that I can still drive it, none of my lights work but it will keep running until I turn it off then it won't start back up again. Its very strange, I tried changing the alternator and battery but apparently that wasn't the problem. I hope you understand what I'm talking about.