I have and have owned an example of every Mopar V8 in pretty much every size, up to the late '70s. I'd stick with the Poly, good engine. Have one now with 103K, stock, excellent street motor. IIRC, the '65 Torqueflite has some 1-year-only features, primarily in the shift mechanism.
I'd be inclined to run a stock cam in a car that heavy, and find or adapt a higher-ratio rocker to get some lift; you'd maintain strong vacuum but pick up HP & TQ. Should be a junkyard or swap meet 4-bbl intake somewhere, and improving the exhaust of course helps.