I do worry about my cats. But the bargain I have with my old man cat, who I've traveled with for the last 5 years is that he tolerates the driving, which he hates, and the campgrounds with dogs, where I do not let him out, in exchange for me tolerating him going out and hunting at night when we boondock. There has to be some aspect of the trip that is fun for him.
Has been an outdoor cat most of his life, and survived one Pitbull attack. He is a very, very cautious animal. The van has his safe space, too. If he doesn't want to come in, he goes up into the undercarriage of the van. No animals can get him there.
My other cats, I don't have as much experience with, so I'm not as confident in their skills. On the way South, I just didn't let them out of the van at all. And not for the first four or five days in our new location, even though there are many other cats and it is pretty safe.