Time to get out the high-speed rotary aluminum cutter bits and make some clearances: wrench :
I've had to do that more than once with big lift cams and Indy BB Mopar aluminum heads Same thing on 426 Hemi heads and aluminum blocks
I had one of the cam grinders tell me that isn't really a problem if the contact isn't hard and bending the pushrods trying to shove them out of the lifter cups or off of the rocker adjusters
He brought up SB Chevy and other motors with guide plates