With a diesel dually I wouldn't think you would mind the weight you're talking about at all. Most of the vehicle combinations I've had over the years probably had less of a power/truck to weight ratio involved. As previous posters have mentioned, large or small they are about the same hooking up etc. and for the most part there isn't much difference towing on the open road, you just have to work with the length and height which can impose restrictions and where you go, and how difficult it is to maneuver.
That said, camping spots available to you tend to go down as the size of your rig goes up. Most of the forest service spots I've hit over the years aren't big enough to park the truck beside the trailer, some even narrow enough to make opening a small slide on one side difficult. If you have slides on both sides you rule out a few more spots, etc. etc. Once you get to 35 foot or so in length and are in the 14 or so foot wide range with the slides out you tend to be getting to a size that limits you from half or more of the forest service/state park type areas in the more desolate areas. RV parks with hookups much less of an issue if any.
