The other thing to take into consideration, as discussed multiple times in the past, the XE cams are just noisy cams. Do some searches on this site and the internet in general and you'll find multiple threads discussing the noise with these cams due to the aggressive ramps.

I have an XE series cam and over the years, I've grown to hate it. I've heard solid cams the are more quite.

I too have adjusted my valves many times thinking there is a problem but, there's not, it's just noisy. When I built the motor, measurements were taken and a set of custom length Smith Brothers push rods were ordered so; no problem there. I've also had two different sets of heads, two different sets of rocker arms, HP manifolds and headers on the car at different times and there was no change with the noise. They're just loud clackity, ticky cams.

In another thread, I can't remember who said it but, he put it best and I'll clean it up; these cams sound like two skeletons humping on a tin roof.

FYI- I run 20w-50; it does not help, it's still noisy.