I'd go through it with a voltmeter when the inverter is running. See if there is appreciably more voltage at the battery than the inverter.
I think your solution of increasing wire size and reducing distance is good. I still think it will give you more bang for your battery buck to ditch the 'spare" concept and just wire both batteries in parallel. This is a perfect example of a situation where having both online would be beneficial. You always have your tow vehicle battery and alternator to get you out of a pinch.
