Galena summit has some very tight switchbacks. I would be concerned when towing a toad in that it is possible the toad could go to steering lock in a sharp turn.
The road from Challis to Stanley is a two lane mountain road that follows a river which has many curves. Not a problem but can be slow and requires close attention in the curves.
You will have to go over Galena summit which does require going up a 7% grade and climb about 1500 feet.
It is not all down hill from Challis to Sun Valley. If fact it is all up hill to the top to Galena Summit then a 7% down hill drop of about 1500 feet to Sun Valley.
In Idaho going North along a river often is down hill and going South is often up hill. Many of Idaho's rivers flow North. The Salmon and Snake are two that do this.