I am not about to jump on the "too little truck" bandwagon, that's the easy way out and I don't think it fairly applies here.
Being new and not having towed something so large and heavy, what you're feeling could be normal. You'll always know you're towing 4 tons behind you, even with a 1 ton truck. Heck, even semi's get blown around by the wind and react to heavy trailers.
There are however big improvements to be had if your towing near a trucks limits. Most if not all common items have been mentioned.
#1 Know your weights...trailer AND tongue weight - using a truck scale. Tongue weight in my experience, often surprises people. And I know many say go right up to 15%, but that has not been my experience towing with 1/2 tons and SUV's. 11-12% has been a sweet spot.
#2 Make sure your WDH is set up correctly with the trailer loaded for a trip. (including payload in the truck)
#3 Tire upgrade. Tires make a HUGE...HUGE difference.
#4 Rear suspension upgrades such as shocks and air bags are a very common, proven upgrades. But note that bags are to be used to mask sag when using a WDH. The WDH takes priority, and sag should not exist when its set up right. The bags will add spring rate and reduce rear end movement.