You may have an issue with feedback voltage to the regulator. If the feedback voltage is low to the regulator, then the regulator will want to kick the voltage up to compensate. That is why grounds are first suspect, but it does not need to be the ground - could be the feed.

Do you have a schematic for the circuit? I would measure the voltages at the voltage regulator. It could be ignition switch related.