Well the panel you received is not the one removed from your car, mistake #1, no telling what condition it was in when "restored", the new panel obviously wasn't trial fitted to the car during the initial restoration process,mistake#2
hope that your qtrs stayed in their "stock" position when the rear was reworked, otherwise no SE panel is going to fit, if there's the chance you can test fit the panel to another "stock" Challenger to see if it's either the SE panel or that your qtrs moved during reassembly, if you can confirm the same problem on an other stock/unmolested Challenger, then you should be on the horn to the individual that refurbished that SE panel, if it's the body work on your vehicle then you have issues to take up with your bodyman, I hope other items where check fitted during the restoration?
How's the trunk side gaps in relation to the qtrs?, on avg it should be about 3/16"-1/4"