2006 Impreza vibration/rattling noise from underneath at 2000-2600 RPMs

My 2006 subaru impreza is making a strange vibration/rattling noise that seems to be coming from underneath me, and more to the front. I can't physically feel the vibration, just hear it. This noise only happens when I am at 2000 rpms and accelerate up to about 2600 at a nice and steady pace. When I accelerate quickly I get little to no noise at all. The noise sometimes gradually wears off after a couple seconds (if I keep the rpms steady in that range). I've also noticed that it happens when the engine is revving down past the 2500-2000 rpm's range.