I would grind just a little more or else you'll need to use a lot of filler to level the whole area. It wouldn't hurt to weld up those large gaps. Then grind down and it will look good.
Are you using one those flap disks? Those are very good for knocking down and smoothing the welds.