It depends on your budget. It is MUCH easier to find an older short E-350 than any of the others. And they are cheaper. An E-450 may carry more weight, but we stock ours full and are still under full weight. Have not had issues with handling.
We are fine with climbing up into bed in our 22 ft rig. It is not that far when we step on a stool on the dinette bench after the cushion is removed.
We love the short length and have taken it on many an AZ dirt road. We use "training wheels" to keep from losing anything underneath although we lost one set of those wheels and the steps on a particularly rocky unpaved road. We do the repairs and go back out again.
We typically go from 3 to 10 miles off the paved roads to find seclusion and have never tried to tow. We fit in some restaurant and smaller parking spaces by backing in and overhanging a spot that is over grass, and in big parking lots it is OK to take two.
But there are folks who tow jeeps and love the extra ability to go on 4wd roads. Could never get as far off road with the RV though if we towed. Maybe taking a dirt bike on a rear mount would offer the flexibility you want and still be short enough to get out of established spots.
I like to take photos of how isolated we are when we camp.

This one is from northern AZ, not far from the northern side of the Grand Canyon.
Have fun shopping!