1996 Vauxhall Corsa Back box/exhaust

Hi, the end of my exhaust which should be attached to the back box fell off whilst I was driving but the back box itself is fine. I quite like the noise it now makes, will it do any engine damage to leave it off or does it need replacing? I have have been told it can alter the engine backwards pressure and damage it but is this the case when the backbox is still in tact and it's just the pipe after the box that has fallen off? Thank you very much for your time, Adam