The "chalk" is oxidation; as the paint essentially dries out from exposure, the pigment basically rises to the surface.
Back in the days of nitrocellulose lacquer and the early enamels, you'd polish that off with a compound.
Problem with the trailer panels is most don't have a very thick coating, so you want to take it easy.
An aggressive rubbing out can remove 1 to 1.5 mils, and there may only be 4-5 mils applied.