I pull several different trailers. Our boat is short (maybe 21 total), a 20 box that is 26' overall and a 28' box that is 34' over all. That one makes us 60'.
Never a big deal...but I don't camp in 'campsites' with the long one. If you're anywhere around 50', I think you'll be fine. I personally don't put any extra effort into the thought process between towing our 16' box, or our long or short 20'. There is more 'planning' when you get up around 60', but it's no different than half the world pulling a huge 5th wheel or what not.
I woulnd't give up the extra space a 20 offers for any perceived savings in agony over a smaller trailer.