This works for antique furniture with worn out holes for caster wheels too.
Over drill the hole, apply glue to a dowel of the same species, tap in, trim flat.
When the glue is try you can re-drill the correct size hole for the caster wheel.
i've also done the exact same process with a hardwood dowel rod in the maple neck on my bass guitar...40yrs of people pulling the neck off for various reasons..
