I can give ya a few downsides:
First off, backing under your camper will be a pain. On my TC, iv got less than an inch in between the outter fender and the jacks. Makes for a TIGHT squeeze. And of course on a SRW, you've got plenty of room.
Another is their size. Like most have said, its mostly mental. Youll get used to driving a large vehicle. Youll also get used to taking a football field to turn around. Going from an extended cab long bed to a crew long bed wont be a huge difference though.
And lastly, maintenance! Youll feel it in your wallet the first time you've gotta replace tires. 4 tires for my F-250 are about $1000. And of course, its closer to $1400 for the dually. OUCH!
No doubt that dually will be tons more stable though!
And if your talking about the 6 liter Powerstroke, yes, they are junk. They'll get you where your goin... after a few grand is dumped in the motor. Ask me! My dad and uncle have one. Both had motor problems. But that's not the point of the thread either.
