Pull the plug! You won't regret it later. (Kinda like Mr Goodwrench saying- "You can pay me now, or you can pay me later!"
)
Worked @ Jasper Engines in the Diesel Block, Head, and Dyno section for 8 years. We had a lot of cam holes that were less then "optimal." Our standard practice was to buff it out first with a wire wheel. Then second, spray it with some Loctite primer, then run a bead of either red or green Loctite around hole and then spread it out with your finger going around it a couple of times. This is all done after its cleaned (hot tanked) and oil free. The heat from the hot tanking helps it cure faster, but if you have to, spray it with some Brake Clean or Electrical contact cleaner to remove any oil residue. Make sure you install the plug in straight, and in far enough. After your satisfied, wipe off any excess on the outside of the plug/hole area.
If you let it sit up 24 hours before any oil comes into contact with it, it'll last you until your next rebuild for sure. Good luck.