Torque shaft removal from trans bracket 2000 XJ Cherokee

I'm trying replace the bushings to fix a rattle on the transfer case shifter and I cannot get the torque shaft out of the bushing on the transmission bracket. Anyone tackled this before? I tried long screwdrivers and a flat steel pry bar with no movement what so ever.