Don't worry about the drive. They paved a big chunk of the Top of the World last year and there is less than 20 miles of gravel. Everybody leaves Dawson in the morning going west and in the evening everybody is heading east into Dawson. I was on it 3 times last year and don't recall any scary parts but I have been on the highway dozens of times. I did tow my fiver from Mandan to Selfridge down highway 6 and that was scary because there was no shoulder and steep drop offs on each side. Any mistakes would have been a disaster. That paved road was more technical to me than the Top of the World.
Most oncoming traffic you meet on the Top of the World will be touring like you and have the same concerns for their own safety. Slow down for each other and enjoy the trip.
