Sounds like a 250 is just what you'll need for the payload. You might squeeze in under the limits of a 150 with small kids, but they tend to grow along with the weight of their toys. :)
Have to try a test drive to see if you find the empty ride acceptable in a 250, I think it's a non-issue. YMMV
Don't need a diesel for that trailer, a 6.2 gas engine will haul it just fine, though it will use more gas empty than an ecoboost.
~~~~~~~~
Just as an aside (NOT to the OP)regarding dry weights, this forum and the guaranteed harping every time someone mentions dry weight. Yes, nobody tows at that weight. Trailer manufacturers also don't sell trailers with YOUR stuff in there, so they can't possibly give you a ready to camp weight. It's an easy exercise to just add 1500 lbs to whatever the dry weight is and you'll be in the ballpark on what it will weigh ready to camp. They are all estimates. I am assuming nobody shops for a trailer by going straight to the scales before you buy it, so give it a rest on using the advertised dry weight to form an estimate.