1995 Ford Mustang Alternator. And/or battery

First the battery light came on and the voltage indicator was in the low range. The alternator was also making a strange sound so I removed it and had it tested at a nearby Auto Zone, where it was diagnosed as needing to be replaced. I also had the battery checked and was told that the battery no longer can maintain a charge, so I replaced both. Initally the car quickly started up, but the battery light remained on and the voltage indicator remained in the low range. I checked all the connections on both the battery and alternator and everything seemed to be in order. However the battery quickly lost its charge and went dead within 10-15 minutes. I had to get a jump start to start it again. While doing this I noticed that the voltage indicator was in the normal (middle) range when I connected the jumper cables to the battery, but dropped backed down in the low range when I removed the cable. I don't understand what else it could be. Is it possible that the new alternator is not working properly? Could it be a short in the wiring? This problem just occupied within the last few days. What else can I do?