First thing you will run out of is Cargo carrying capacity (payload). That is the capacity you have to carry the family, the dog, cargo in the vehicle, weight distribution hitch, and trailer tongue weight, combined.
As you load people and cargo, your payload drops pound for pound. Your tow capacity goes down with it.
You can calculate what trailer weight will max you out, by subtracting weight of everything, everybody, and the WD hitch, from your specific payload. That will tell you (available payload) the trailer tongue weight that you can carry.
Divide your available payload by .13. That is loaded trailer weight that will max out the tow vehicle, on payload and GVWR.
I suspect, with six people and a big dog, you're pretty close with what you have.
Check for a loading info sticker, on your drivers door post. That should tell you, your cargo carrying capacity (payload). Subtract from that number, the weight of everything and everybody you plan to put in the vehicle. Then subtract another 100 lbs for a weight distributing hitch. What you have left is payload available for the trailer tongue weight.