We have a toyhauler, so I suspect my camping spots may be different that just a normal 5th wheel trailer. Having said that I was towing my 9900 pound bumper pull with a 2009 GMC 2500hd Diesel 2x4. We were trying to tow it up a gravel hill in Munds park AZ. There was a turn at the start of the hill and being the first time I was towing my new trailer I probably didn't get enough of a head of steam. Got stuck about 1/3 of the way up. I had to get an entire line of trucks/RZRs/motorcycles to move while I back the trailer downhill. It was quite a pain.
That plus our desire to go to the dunes someday, and the desire for a larger 5th wheel made our mind up. My wife was the main proponent of getting a new truck and making sure it was a 4x4. Sooooo 2015 Ram 3500 DRW 4x4 was the choice for us.
I haven't needed 4wd since we purchased the new truck. But, after the initial purchase (price) I don't really feel any downside. It's there if I need it and I can't conceive of a situation that I will ever wish I didn't have it.