Trouble removing brake drums

So, I was told by a local "chain" repair shop that my rear brakes are bad and need to be replaced. I'm familiar with Car repair so I want to save the $150 in labor and do it myself.

I got the drum on one side off without much trouble. The problem is, the other side I can't get to budge. In the repair manual, it talks about removing the access plug, inserting a screwdriver to lift the adjuster lever off the adjusting wheel and using another screwdriver to turn the wheel.

For the life of me, I couldn't get the adjuster wheel to turn in the direction that contracts the shoes. Any suggestions? I tried it for about 2 hours today and didn't get anywhere.