Often when driving the seal the garter spring pops loose.. When this happens there is no pressure holding the seal against the shaft... Pack the cavity on the back side of the seal with grease before trying to drive the seal..

As mentioned a good driver will help getting the seal driven squarely, I've used the proper tool but also have improvised with a large washer & the top of a jack stand, a large socket & extension.. What ever you use it needs to drive the seal very close to the outer edge, if your driver is to small the sel will distort & fail..

Also if your end play on the bearings is to loose it will allow the bearing to be pushed off center due to the tapered design of the bearing..

Get after it!!!

While writing this post you posted.. Your friend is correct, the adjuster is on the passenger side & the end play is set for both sides with one adjuster...