Maybe they don't have any bite. Could be glazed pads or glazed rotors. If you didn't bed them in correctly then you could have very low friction and not much stopping force.

You could have a variety of issues with the brake system but the overall design should work fine. You just need to figure out where the problem is.

The 11.75 rotors will work better on a big heavy car than the 10.90 rotors, but you should still be able to get the 10.9 system to stop the car effectively.