I appreciate everyone's responses so far. Putting ballast on the pin would not be my first choice, just wondering if it would work. As well as wondering how much I am putting on the axles being a little low in the back.
My pin is really light.
Pin garage empty: 2,620 lbs (17.75%)
Pin Bike+water: 2,140 lbs (13.16%)
With the bike and water it tows absolutely wonderful. Without the weight, it sways a lot.
I was restricted by length due to my choice of camper storage. As a result, I chose a double axle trailer under 38'. I knew there would be limitations but this model seemed to have the best CCC of the ones I looked at. Once on the scales, obviously I am having some issues with distribution. It's fine for my cycle and smaller sxs, which is what I will have to stay focused on. I will play around a little more, run less fuel in the cell. Move some weight around the best I can, maybe see if lowering the pin will help any. The factory spare is just in front of the axles, maybe I can move that to the front storage as well.
Upgrading the axles seems like the best option, but not necessary if I can figure out how to distribute the weight.
I'll keep everyone updated.