If it's the original engine I'd keep the 340.
Hemi swaps are nice but doing it on a low budget isn't going to happen. I've done four of them and you should expect to spend between 6 to 9k minimum.
I know it's e-body related but here's a link that will give you some insight on the hemis.
https://board.moparts.org/ubbthreads/showflat.php?Number=3930520


'86 D150 -'03 5.7 Hemi '72 Charger - '08 5.7 Hemi '73 Barracuda - '08 5.7 Hemi '73 Challenger - '08 5.7 Hemi