1993 Acura Integra cold start and Idle problems

When its cold out, my car wont start. Itll crank fast at first and wont turn over at all. After a while the cranking gets slower and slower then eventually the car just dies and wont even start. When I jump the car with jumper cables itll start up just fine. The weird thing is that once its started the engine will get warm and idle really low with the check engine light on. Ive already changed the oil and changed spark plugs. The other thing is that when the motor is still warm the car will start up fine. Ive been hearing squeaking noises from my alternator belt so I just went and bought a new belt just waiting to install it. I have another problem also, when I step on the gas the lights brighten up but when I let go of gas the lights dim down. Please I need some help, my car has never had this problem before.