Ditto what joeshmo said regarding primer not filling 150-grit scratches.

In addition, do not attempt to fill cracks in the paint with primer either. Keep sanding until the cracks are gone, even if it means sanding all the way to the metal. If it comes to that, there is undoubtedly rust that needs a good sanding when you get that far anyway. It is only necessary to go to the metal if the cracks go that far, and only in the areas where there are cracks. In areas where the paint is in good condition, just give it a good sanding with 320 and move on.