For normal driving only with a romp once in a while and if you already have all good enough parts for std stroke?
I wouldn't.

If you need all new internals anyway, a little extra displacement can help offset the big ports and valves and give it a little more low speed snap.

I went mild 572 on the one I built for the GTX, but have a more street/strip mindset for it and started with nothing, and while I do care what numbers the car puts out, at the same time I have to admit the hemi is just as much for cool factor as anything else.

Have tentative plans for another one that is a cruiser, and there's no reason to increase the displacement at all.