There are reinforcing plates you can add inside the door under the mirror.
They greatly increase the mirror's rigidity and keep the sheet metal from flexing.

Joe

Go to Mega Parts,- body -mirror reinforcements
P.S. even more important, door handle reinforcements