I think part of the problem is the brackets are slightly deformed.
I went the cut and weld route and just finished one side. Needed to persuade the bracket into shape but it went very smoothly and after a little clean up with the sander, prime and paint it looks great.