I agree with your approach and what you are looking for is a transfer switch. In rigs with generators, the transfer switch automatically powers the rig from with either shore power or genset.
I chose not to light up the whole rig from my inverter and used a single circuit 15 amp transfer switch instead. I have a 600W inverter and selected the circuit that I would want powered from the inverter. The transfer switch is hardwired into the breaker panel and connected to that circuit, so only the outlets on that circuit receive power when the inverter is powered.
Here is the inverter and transfer switch set up
The Xantrax has a remote option so I mounted the remote in the trailer by the other controls, along with a voltage meter: