You do need the inverter close to the battery. The charge line from your truck may be able to keep up with the running power needs of the fridge, but it is entirely insufficient for the starting surge.
Either moving the plug or installing a DPDT switch as a transfer switch would be safe and workable. You could also leave the fridge connected to the inverter all the time and rely on the converter in the trailer to supply adequate 12V power when plugged in.
You need a pure sine wave inverter that's around 1000 watts or larger to operate the fridge. A typical modified sine wave will not play nicely with the compressor motor, and a smaller one will have trouble with the startup surge.