Starter Haning

I have a 71 Nova with a 350 and a saginaw 4 speed. I have had the same problem with about 5 different starters now. None brand new. When I put the starter on it will work fine at first, maybe sound like it needs a shim or two but still turns the engin over pretty well. After I have started the car a few times the starter will start to grind real hard and wont turn the engin over. So I will put some shims in it untill it sounds and works like it should. Then after I turn it over a few different times, it does it again. Then I will go through the entire process again, but I still have had to luck.

P.S. I had a turbo 350 transmission with 2 different flyweels on it before the 4 speed and had the same problem. And one of these 2 flywheels were brand new. Still had the same problem.