FWIW,
For about 10 years, we had a big 5th wheel that (in my view and per my math) required a dually. So my daily driver was a dually F350 and the wife had a Prius.
Fast forward to our now non-RV days: Current fleet includes a dually F350, dually with dump bed F550 dump truck, a SRW F250, and the Jeep Liberty that has replaced the Prius. I still routinely (not daily) drive the F550 and F350. The width makes it a PITA compared to the Jeep Liberty, but the handling challenge comes from the length more than the width.
Would I rather drive the Jeep Liberty around town? Sure. Would I rather drive the Jeep Wrangler on the trail. Sure. ...but if I have a big towing job and the dually is the right rig, I wouldn't get an SRW just because of the "dually" issue. If I could get by with a 6'6" bedded SRW, I'd certainly go with that vs. a dually, but if the choice is long-bed SRW vs. long-bed Dually, the handling issue will be created by length, not so much by the hips.
All IMHO anyway.