I just did that trip with my 35 ft fiver, but came down from Durango to Santa Fe. I took Hwy 84 between Downtown Pagosa Springs and Santa Fe. Was very nice and no problems. I recommend Aspen Rose RV Park in Durango. Watch the steam train chug by on the other side of the highway. Get CG reservations early.
* Depending on when you go, You might want to start watching the weather in Durango - in the 30's at night now, And Silverton is at the 9500 ft level. You will go over a 10,000 ft pass to get to Silverton.
