I've sprayed cars in a booth and gotten those specks. I even wet the floor down with water, used an elaborate filter system in a downdraft paint booth and wore a spacesuit looking paint uniform and washed my airhose to keep as much trash out of the air as possible, and still got those dirt specks in there. You may be able to buff them out, and maybe not.

Try wet sanding and see if they go away. The trick to wet sanding is to realize you're only trying to take off a miniscule amount of paint in order to make the surface flatter. If you hear a squeak, you've got dirt in there and it's time to rinse your paper. It takes a few times to master it. A squeegie is a valuable tool also.