You are losing the ground. Run an auxillary small 16 ga ground from the armature field to the body. Reason it worked is because you wiggled the ground.