Yes, high summer is the only time it probably won't get below freezing at night. Swimming in the river isn't recommended either, as the temperature is such that only a few minutes is a recipe for hypothermia. Yellowstone in the summer is almost a constant flow of traffic. There is not much in the way of isolation in the summer. In fact trying to find a place to stay could be taxing. I've been there once in mid-June and swore I would never go back during the summer, and never have. We go early spring, and mid-to late September.
I really don't think you're really aware of the distances involved in and around Yellowstone.