I have a 28x46 building with two 10x10 doors which I use for a shop and race car.
The 10 foot door is narrow for a trailer or to open the truck door when half in the building. I would recommend a 14 foot wide door to allow getting in / out of the truck, walking around the trailer, etc when half way through the door. Max height is 13'6" in most places so I'd also look for a 14 to 15 foot tall door to allow any RV or commercial truck to fit.
I'd also allow 4 feet plus all around the RV with the slides out if you plan to load, unload, clean or service the trailer while it's indoors. I think this puts you around 20x40 without the truck or 20x55 with the truck attached, not allowing for anything else in the building. If you're strickly using it for RV / truck storage, you can get away with a smaller space.
A design option to consider is an extended (15 to 20 foot) overhang with lighting above the garage door to allow covered (but not indoor) hitching of the trailer. The overhang most likely will not count toward your total square footage.
For the utility trailer and lawn tractor you could also build an L shaped building with 20x40 for the RV and 20x20 for the utility trailer and tractor, add a basement (spancrete) for the utility trailer and lawn equipment if the slope of your yard would allow it (side or rear entry) or extend the depth of the 2 car attached garage with a door in the back.
Your X340 will likely haul the utility trailer around the yard (assuming firm ground and gentle slope) but I'd be sure it was empty and careful of the tongue weight.
The most important thing is to consider how you will use the building and design it to fit your needs. There are many options when you're building from scratch and it also doesn't hurt to be creative when working around the constraints imposed on you.
Jeff Graves
10 GMC 2500 Crew Cab 4x4 D/A (tow vehicle)
00 Mazda Miata (daily driver)
94 Honda XR650L, 03 Suzuki SV1000S (toys)
Miska 18' open car hauler
and a couple Miata race car projects