No, you shouldn't have to remove the axles to get the drums off. Make sure truck is in neutral, parking brake OFF. If you can rotate the drum easily, then the brake shoes are not sticking against the braking surface of the drum. Then put transmission in park.
Most likely the problem is the center hole of the drum is rusted on the end of the axle. Removing the axle would allow to hammer on the inside edge of the drum, which might loosen it, but it's a lot more work. If you don't plan on reusing the drum, you could try this, I did it to get a stuck front rotor off of a '96 C/K 1500: Take a good 1/8" drill, and drill several holes into the drum right next to the axle. Make the holes close together. Then use a hammer to tap it loose. Other than that, you might use a hand held propane torch to heat that area up a bit and then try hammering it loose. There's really no other way that I can think of. Good luck.