Voltage regulator is grounded to the body, alternator is grounded tot he engine. The problem you are seeing is the ground between the body and the engine is bad.

Run a quicky jumper from the alternator case and voltage regulator and see if everything is as it should be.

They say there are no such thing as a stupid question.
They say there is always the exception that proves the rule.
Don't be the exception.