A properly set up trailer should not sway without any sway control under normal towing.
Sway control is needed for the not so normal towing conditions such as a large motor home passing at 20 mph faster than you are going and only 18 inches from your side mirrors.
If you do feel the sway control there is most likely a sway problem that needs corrected.
Same for a properly setup brake controller. When braking under normal conditions you should not feel the trailer.
If you feel the trailer pushing or pulling then the brake controller is probably not set correctly.