I found myself in that position once a few years ago, towing a Jeep Wrangler behind a one ton dually carrying an 11.5 foot TC.
Like the OP, I did not want to disconnect, so I attempted to back up.
After about 4 or 5 feet, the Jeep steering SLAMMED hard over against the stops.
Luckily, the truck was a 4X2, with the sharper turning radius than a 4X4, so that was just enough that I could straighten the Jeep wheels and drive off.
I vowed then; "Never again!". The few minutes required to disconnect the toad (or unload the dolly, disconnect it, etc.) and reconnect are most assuredly worth it, IMO!
Yes, I know, many have said that they back up a toad all the time with no trouble. Good for them, I will not try it again.