Did you try chilling them? On some projects with press fit bearings, I've been able to pop them in by hand if you get enough temperature difference, and the clearances seem to work out better.