We went from a toy hauler to a truck camper with enclosed trailer. With just two of us, it has been a big adjustment due to the lack of floor space. We do like flexibility of the TC with detached garage, but will be looking at the mega TC's once I have a bigger truck. The weight adds up fast - 6K+ lbs for the largest TC's when filled with water and personal items before you add the weight of people, towing tongue weight or extra items stored on a flat bed or utility body.
With your family, look at going to a F550 or 5500 immediately so you can get a 12' triple slide TC that has enough room for all of you. If you decide you want to take something in tow later, you will have the capacity. Surprisingly, the class 5's are not much more than class 3's. You can get a comfortable crew cab gasser for under $50k new or a diesel for less than $60k but expect to add the cost of a $3k-5K flatbed to the rear.
