Dimming is normal for cars with the small case alternator but the big one you have should not have that problem. I'm running the big one on a mostly stock wired car. Even with a big fuel pump and lots of electronics idle voltage is rock steady. Lights, brights, AC it doesn't matter.

In addition to the suggestions so far, make sure the alternator case has a heavy ground wire. The 100A alternators are mounted in rubber bushings and need the wire to head or block to get a good ground. There should be a stud on the case for this purpose.

On the ignition switch, I guess you could have a problem, but if the alternator was putting out enough juice, it shouldn't matter if its at idle or not. I would start with the alternator/regulator/associated wiring first. Also, are you running a standard diameter crank pulley? A samll pulley could cause the problem.