2001 rear brake noise

I recently bought my 2001 wrangler with 64,000 miles. It has been lifted and larger tires and wheels are on it. I just did some routine maintenance that included rotating the tires. When I was finished and had all 4 wheels back on the ground, I drove a short distance and noticed a grinding brake noise coming from the passenger rear wheel. Thinking something could have gotten dislodged inside the drum while rotating the tires, I came home and took that wheel and drum off. I could not see anything out of place, so I cleaned it up really well and put it all back together. It still makes the noise. The strange thing is that it only makes the noise at speeds between 10-15 mph and while slowing down, not while speeding up. The brake does not have to be applied to make the noise, but it is definitely a brake type grinding noise. The shoes and drum are in good shape, and everything else seems fine. I am stumped.