the length of the wear on the lifters seems to correlate to the amount of cam lift from the bottom of the lifter bore upwards. check the bottom of all the lifter bores for sharp edges. you mentioned that a new lifter falls right through. you could use a snap guage and a mic to measure the bores, or a burnishing ball as a go/ no guage guage. do other lifter bores show the same wear, even if it's not as bad as the one in the picture?
anyway, normal side loading in a properly sized bore without sharp edges shouldn't have those issues. check size, roundness and look for sharp edges. i bet the root cause of the problem becomes evident very quickly.