The 04 gens are going for stupid money & don't know what it'd cost to rebuild your Cummins, but if you need 4WD, I'd keep it. I loved my 02 2500 4wd, but the had to learn the hard way what upgrades I shoulda made!.
The 98-2002 gen 2500's with a steering box stabilizer, 727 upgrades for towing & the Fast Pump upgrades are hard to beat. I like the later gen's interiors & front suspension, and they are much quieter than the earlier gens, so that's something to consider.
Two friends of mine just bought 04's. Both 4wd's & the one with 23K orig miles went for $29K & the one with 120K went for 23K. I think your decision basically comes down to cash outlay to fix the 04 & do you really need a 4wd. Best wishes in whichever direction you go.