I'd start with getting the tires checked, rotated and balanced. Although, I would expect the problem to get worse when you go faster.
Once you verify that the tires are good and balanced, I would check the ball joints, cv joints and u-joints. Make sure they don't move when you shake them by hand. Also, check the calipers to make sure they are tight.