You do not need to remove the door to take the door panel off.
Here is a step by step of how to remove the door panel.
Remove the 2 phillips head screws located just above the interior door handle.
Remove the inside door handle cup
Start in a lower corner and work around the panel and gently pull to separate the front door trim panel from the front door to disengage the white grip pin-type retainers.
** If you brake a white grip pin-type plastic retainer don't worry about it, you can by some at any parts store.
Lift the front door trim panel to disengage the top lip from the weatherstrip.
**It's a little easier if the window is down
Separate the power window switch finish panel from the door trim panel.
Release the locking tabs. **Be careful when the power window switch module is released from the panel you don't break any wires.
If equipped, remove the courtesy lamp bulb from the lens and remove the front door trim panel.
I hope that helped you out. If you have any problems just message me and I will help you out.