I had a 4 wheel drum PB car. The brakes had good initial bite but weren't very even left/right and front rear. Also they weren't good at speed.

Its mostly the fronts that need upgrade. There are lots of ways to do it. Check out Mopar Action Tech pages. You'll find a step by step guide on how to use standard MOPAR front disc brakes from later years. Master Power Brakes also sells complete kits with stock type parts. Either setup should dramatically improve braking performance.

Much bigger front brakes and rear discs are available also depending on how far you want to go.

Staying with manual brakes will work with the upgrade - just make sure you keep master cylinder diameter on the small side so pedal effort isn't too high... or upgrade to power if you prefer.

Most likley you'll need to change the front spindles so the level of difficulty is similar to a front end rebuild.