My first thought would be that the lifters may want a high zinc oil since there are sliding contact surfaces vs. rolling. Contact stresses are way down compared to a needle roller and should see wear unless something is way out of whack. Another thought would be if the oil is not getting to the roller. And that can open up another subject and can of worms.

Even though there is a lifter bushing with a oil hole, if you are using a mopar bodies lifter meant for a stock bore you may be restricting or cutting off the oil supply to the roller. Typically with a bushed bore, you would use a chevy style bodies .903 lifter with an oil band to catch the oil from the oil feed from the oil bushing and supply the bushing on the roller lifter.

AG.