When's the last time you changed the oil? You say it's not a daily driver. How long does it sit? Did it sit for a long time after the last time you changed the oil?

My guess is that you changed the oil and put in whatever was on the shelf at Walmart. This oil was one of the SN or slightly older oils with lower levels of antiwear chemicals (zinc). Then the engine sat for several weeks so the next time it started pretty dry. It doesn't take that long for the cam and lifters to commit suicide.

The bad news is that all the material ground off the cam lobes and lifter bottoms made its way into the oiling system and now you need to clean everything.

R.