The reason the cam follower comes off is usually due to the valve seat coming loose in the head, not the lifter. I cant say that I have ever seen a bad lifter in one. Have also seen a couple of broken valve springs, worn cam lobes and follower rollers. You are just gonna have to take the valve cover off and take a peek.