Here is an update, as I think I am going to move on from worrying.

I put about 100 miles on an oil change, driving it pretty normally/spirited. Dropped the oil and the magnetic plug had a slight dark residue on it. Super fine, but obviously magnetic. Cut open the filter and it looked way better than prior, still some of that slurry stuff in the media. Zero flecks of any silver bits like prior time.

I borrowed a borescope and went in through the drain plug. This borescope has a super bright light, which was great, but was too large to fit through the dipstick tube. Was hoping to see the pickup inlet area, but that wasn’t feasible from the drain plug, couldn't bend around to that angle. Checking out the entire pan in detail, could not find anything laying down there, except one tiny fleck of silver on top of the pickup tube that reflected in the light. Pulled the distributor and inspected the cam and pump pushrod and surrounding area, which all looked normal. Did a few passes in the camshaft valley and along the lifters to see if there was anything laying up there, could not find anything out of the norm.

So I am just going to let it be and keep checking on my oil change intervals. Probably by the time anything comes to a head, it will be after I am around to worry about it.