a word of caution when drilling brass set screws, or enlarging holes that are already there. this applies to all methods, drill motor, drill press, or hand operated pin vise.
watch the "break through" at the end of the hole. this is where the majority of broken drills happen, because the flutes of the drills stick on the last couple of fines. this happens whether using lube or not, and is just the nature of the material.
it is also a good idea to replace the bit size often, so you have a sharp cutting edge.
most think brass is a soft material [and it relatively is], however, it is abrasive, and requires the sharpest tools you can use to successfully cut, machine, or drill it.
just speaking from a 45+yr working career as a machinist, [and still practicing] with lots of scrapped brass and tooling under my belt.