cap the vacuum line to the booster and see if it still does it.
it will be harder to brake in general, but if they are grabby because of the booster, it will stop.

there is an internal valve inside, that is supposed to be gradual, but if it is sticky you get what feels like you are slamming on the brakes once it breaks loose inside.

without the assist it would be gradual once you bleed off all the stored vacuum pumping the pedal a few times.