We used Torrey as a base came in 2014 and made a day trip to Bryce down Scenic 12 one day.
As an newby back then I was impressed with the road and glad we were just in the TV. However, after pulling all over Colorado in 2015 I would pull the trailer down 12 if I needed to.
Just take your time. We went south on 12 at noon on a sunny day and as we came down the mountain closest to Torrey we came around a blind curve and all traffic was dead stopped. Cows had gotten out of the pasture and were on the road, and the cowboys on 4-wheelers were trying to round them up.