Loud rattling when starting the vehicle

When I start my car, makes this loud rattling noise. When I put my car in reverse or into drive the rattling will subside. But it will continue (sometimes getting worse ) until I hit 10 to 15 miles an hour.
I know basically nothing about cars other than how to change my own oil and change my tires. This morning I had my muffler fall off, thought it would kind of fix the problem. It didn’t. However the pipe that your muffler is attached to (exhaust?) If I push on it or stabilize it while the car is in park the rattle will stop I just don’t know what it is or where to start to fix it.