ABS sensor rubbing after replacing CV shaft

Hi
I just replaced the CV shaft on the front of my GC and I started getting a clunking noise. IT seemed to be about one clunk every revolution of the tire. After removing the abs speed sensor I seen it had rub marks on it. So I kept it off and hung it up out of the way. Now the clunking stopped. But my abs light is on now and I have no idea how to fix it other than start grinding down the sensor. But I do not want to ruin it. What would cause this.