I hope I am never anywhere close to you with my family aboard, while your towing along chawing on jerky, which is inattentive driving. Oh the humanity.
On a serious note.....don't get caught up into the myth of " I dont even know its back there". Well it is back there, and your going to know it. When someone posts " I dont know its back there" is usually when I stop reading. Thats just forum BS. I can tow a 750 lb log splitter or mortar mixer with an F450 and know its back there.
Feeling jerking and other feedback is normal. As long as your not developing sway, especially going downhill while coasting, or not getting some type of excessive jerking or bounce all the time to the point your feeling it hard in the steering, go with it.