It sounds like the alternator is charging with full field current. This could be a bad voltage regulator, or a short in the wiring that goes to the voltage regulator.
#1. Unplug the triangle connector at the voltage regulator, start the car and rev the engine to verify the alternator is NOT charging.
#1a. If in above test, No charging is observed, replace the voltage regularor.
#2. If in the above test the alternator is still overcharging, disconnect the field wire going to the regulator and repeat the test in step #1.
#2a. If No charging observed, check wire for short to ground.
#3. If alternator is still overcharging, there is a short at the alternator. Usually this is just a short at the alternator brush holder.

The $15 voltage regulator is fine if it is the bad part.