As you mention in your post, I think you need to be as concerned with payload as much as anything. It looks like your family weight could be as much as 900-1,000 pounds in the life of this vehicle, and that's before you put anything in the cab or the bed of the truck. You could easily hit 1,200+ before you ever hook up the trailer. A 5,500 dry trailer could readily be 7k loaded. That's going to put you somewhere in 800-1,000 range for the hitch.
All in all, I would suggest you need to be ready to carry 2k+ in payload. I'm more familiar with he SUV market, so others may know better, but I think that's tough to find in a 1/2 ton crew-cab. I'd suggest 3/4 or 1-ton to give you the power and safety to handle the distances, climbs and weights that would come with the trips you expect to take.