Put a stiff wire wheel in a drill and keep the rpm really slow. If you go full throttle the rust glazes over and then gets even harder to remove.

You can also use a dremmel tool with a sanding disc or small grinding bit and dig it out that way.

Clean the area with some ppg metal prep after your done.