My Street Hemi had an oil consumption issue when fresh even though it had been rebuilt by a reputable Hemi SS racer. Turned out to be 2 problems: it was way rich, washing down the bores (you could literally smell gas on the dipstick), and oil was getting past the guides. Correcting the carbs fixed the 1st issue. Installing oil seals on the guides that were high-heat silicone (got them from Silver Seal) cured the 2nd. After that the car ran cleanly even with a fairly big cam, and quit drinking oil.
I agree on the geometry thing; you can see how the springs that kind-of locate the rockers allow side movement; Landy came up with a collar arrangement back in the day that fixed that.