Hard to pick up speed

Just recently it started to act strange when I would try to pick up speed. It will be just fine driving at a low speed, but when accelerating to a speed for the interstate or going up a hill, it seems to have a hard time reaching the desired speed without having to press really hard on the gas pedal. I don't know much about RPM's and whatnot on a vehicle, but I have noticed in the past when the car didn't act strange that the RPM's would go up a little, past the 2, to allow the vehicle to get to the desired speed, and then would drop down and stay around the 2 mark. It seems like even at a highway speed, the RPMs will hang around just above the 1. The RPM take a lot of pressing on the gas to get up past the two, and to get the speed to reach above 40 mph. Almost like its stuck in a certain gear. Is this a transmission problem, or what could this be. All I know is, it doesn't feel right. I would hate to push it to its limit and have the engine lock up or something when I try to keep up with the flow of traffic.