Caliper brackets are grooved pretty badly.

Rather than grinding metal away, I proved to my students you CAN weld cast iron. File the area clean, then prop up an acetylene torch to heat the area for a good 10 to 15 minutes with a medium flame, then use a wire feed welder to fill in the grooves. Grind the surfaces down after the weld cools.

If you grind metal away the pads will rattle too easily. They will click too each time you release the brake pedal from the spring clips that go into the pistons. If you have to resort to that, just bevel the edges of the grooves so a pad doesn't catch and fail to fully release. You can check the auto parts stores for repair inserts but if the grooves are that deep, the inserts are just going to bend under hard braking.

Also, be sure to use some high-temperature brake grease on those slides when you put it back together. Rusty Lube is one trade name I'm real familiar with. It is copper-colored, but there are others that are mostly black. They all contain molybdenum disulfide.