The difference is small, but
1. The larger roller is in a different place on the lobe at the same crank degrees, because it doesn't contact the lobe at its highest point (like all flat tappets), but at its closest point (tangency); think of it as a clock position
2. At several points of rotation, the larger roller's speed is higher, but not linear or proportionate
3. A concave flank will not like a larger roller at all, and may skip over the root of the reverse curve, then get lofted by the far side. A smaller (than the designed config) roller will simply act differently without damage

A strong enough spring (and/or low enough RPM range) may keep the bits in contact, but a SpinTron will show chatter at (perhaps) more than one point in the RPM range. If the engine must be run in that range severe damage may result.