If you are used to driving the roads of the southern Appalachians you'll have no problems. Clayton to 17 is curvy and has steep grades making for slower speeds and lots of work on the steering wheel, brakes and accelerator- but no show stoppers. But as you know the southern Apps and the Blue Ridge are awesome making it all worth it!
180 is fine. (but don't do Brasstown Bald in an RV)
Agree its not as bad as 64--- but heck it is the mountains! 19 or 17 into Helen can be curvy and steep also. I did them last week in our 38 foot gasser towing the dolly and toad. We survived but not going back for a dose for a few more weeks :)
If you need groceries or fuel before going in to Vogel you may want to stop in at the Ingles in Hiawassee.