My passanger side rallye mirror's glass on my 71 Duster is loose, more so it's loose on the post that it moves on, a good bump or shutting the door will make it move.

Short of buying a new one has anyone fixed this issue before?

Hopefully, it's a chrome mirror and you can do this without ruining the paint. Soak the whole mirror in a container of lacquer thinner to make the adhesive holding the mirror glass turn loose. Then take the screws out that hold the assembly in. You will see where the solder has come loose on the plate that holds the mirror head tight against the ball. You will have to drill and tap the broken part and use a small screw to hold it tight. Kinda hard to explain, but you will figure it out when you get it apart. I fixed a passenger side Challenger mirror that way, and it works fine now. Good luck.

That sucks because of course it's painted.

Often you can reach the screws with a long thin screwdriver.

Additionally you can try heating the glass to soften the glue and remove it.