I am leaning toward the possibility of having number 3 & 5 spark plug wires crossed. It is very easy to do and they are next to each other both in the firing order and on the engine so at idle they will somewhat fire but cause a somewhat ratty idle (like you said you had) but under load it will have definite roughness along with possible backfiring. The miss firing would/could worsen as the plugs loaded up.
Check your firing order, not just those 2.
Also check for vacume leaks.