Well, it's supposed to hold vacuum long enough to make at least a couple of engine-off stops. If the check valve and its grommet are good the diaphragm must be leaking.

Can you hear any leakage immediately after the engine is shut off?