Maybe the hub of the damper is tight on the crank and the outer part of the damper is loose from the hub?

Since you said you still had to use a puller to get it off?