Brakes failing in reverse only

Do you have rear disc or drum brakes? My guess is you have drum brakes and a lining has rusted off its frame. That can allow it to rotate backward with the drum, and away from the frame. That leaves a gap of as much as 1/4" between the shoe frame and the drum. There is not nearly enough pedal travel to make up for that much shoe movement.

To verify this, come to a stop while driving forward. Shift to neutral, then release the brakes. Shift to reverse, then apply the brakes immediately, before the vehicle starts to move. The brake pedal should remain high and normal. Now release the brakes, then reapply them as soon as the vehicle has moved a few feet. If the lining has shifted position, the brake pedal will be low.