Stuck or sticking brakes on the trailer can cause a noticeable pull, if stuck check the trailer brakes on the same side it is pulling to.
If pulling only happens while braking then might have inoperative or badly adjusted brakes on the opposite side from the direction of pull.
It is also possible your vehicle may have a stuck or sticking brake pads in the front caliper. Sometimes does not give a noticeable pull until you start having to brake harder like under a heavy load of towing. With owning many different Ford trucks over the yrs it is a constant battle to stay ahead of the sticking brake pads..
The manufacturers make the vehicle brake pads a bit too tight of a fit in the calipers, add in some rust and now you have hanging/sticking brakes.
I like to file/grind the pad ears until the pads move easily and smoothly in the calipers, takes care of that issue until pads need changed.