Shaft can get corroded, even slightly. Or even dirt build up on exposed shaft.

Also the thin film of oil on shaft dries/runs off.

Then shaft moves and swipe seal is damaged.