I've been running a 2500 6.4 4X4 for about 3 years and am up at around 55K miles. I tow an 11K 5th wheel out west with lots of hills and have the less than ideal 3.73 gears.
The towing is 5% of the truck's life at best. Daily driving the other 95%.
Daily driving I give the truck an A-. Towing my load I give it a C+. I suppose since I'm basically maxing the truck out at every number I could be a bit more generous with my towing rating.
Annoyances towing are all with the tranny. The gearing of the 66rfe is borrowed directly from the 68rfe paired with the Cummins (diesel) motor and are really not meant for a gasser. The gap between 1-2 is a MILE and not suited for a gas motor that makes all it's power above 3500rpms. There's kind of a no-man's land around 35-40 up a steep winding road. Your choices are wind it out in 1st @ 5000rpms or let it hit second only to slam back into first. When I get caught in this situation I usually just slow down and ride out 1st at about 30mph. The good news is the cabs on these newer trucks are insulated extremely well so you really don't hear the engine scream like with older trucks.
In the end, I'm doing everything "wrong" - max weight rating, less than ideal gears and in the mountains out west... and I'm surviving. Financially, I could afford a diesel but I just don't tow enough to justify it. I've never felt the least bit unsafe with my setup. It's just not what it could be. Compared to trucks 20 years ago I'm flying first class with a glass of champagne in my hand :)