I commend you for think this through. Do not fall into the upgrade trap. Buy your lat RV 1st.
Since you have towing experience with 18 wheelers. Towing is not a factor.
The debate is space and comfort vs. finding a campsite.
I towed a 36' TT prior to my current 5'er. I was always able to find a sight anywhere I cared to go. Nevertheless there were always more smaller sites and there were always a few sites I could not fit into. In the end I always found a site. In 5 years I was never turned a way or left a CG because they did not have a site large enough for me. Keep in mind I'm on the east coast. I use both private and public CG's.
As to space I prefer to have ample room. You mentioned you have a tall family. Beware the kids will outgrow the small bunks in some of the smaller units.
There is more to the equation than just a place to sleep.
I also prefer the kids to have their own space. A separate room gives the kids a place to store their stuff as well as a place to hangout. If the kids do not have their OWN space, than the adults Do Not have their OWN space either.
Lastly, consider tank capacities and cargo carrying capacity. Being able to haul and store fresh and waste water is important and are factors often overlooked.
There will be compromise. Site selection maybe tougher for a larger rig, on occasion.
However having enough space is a factor every trip. The kids will only get bigger and bring along more stuff including friends. Pay attention to the physical size of the bunks and decide if you kids will fit in them for the next 10 years.
Choose a size and floor plan with the future in mind. Avoid upgrading for at least 7-10 years.
If you follow there forums you will read of a lot of upgrades. We started with X and moved to Y and then on to Z. Many slowly upgrade every few years to gain space and features. Buy your Z model first avoid X and Y.
Cramped and spacious can be very subjective terms. However adequate and enjoyable are not. If the space is not adequate the trips will not be enjoyable and the RV will eventually sit idle.
The only way to get value out of any RV, in any size or type is to use it.
Be sure to pick a size that will be usable not just right now, but for years into the future.