You can remove them fairly easily (but carefully!) by using a drill of the correct size. It will take the head of the insert off and the back of the insert will fall out. Ideally you want to be careful not to let the drill bite and spin the head which may scuff the paint, but this can be done and if you are repainting not an issue. But I agree, no reason to remove if they are in working condition. (I needed to do this because one was incorrectly positioned...don't ask...)