Need more information to be able to give good advice.
What converter do you have? Some are "smart". Most are "stupid".
Suspect any inverter large enough to handle your RV (all but heavy loads such as roof A/C, water heater and any other heavy loads) it will have a smart built in charger that is likely vastly superior to your converter.
If true, there are two good choices: Disconnect or remove the converter. Leave converter in place, but install a switch on the 120 VAC hot to it. This would give you the redundancy if the charger section of the inverter/charger failed, but allow you (AS YOU HAVE TO) to make sure it is disabled when not on shore power. If left connected, as posted above, you would quickly deplete the batteries-- inverter powering converter powering inverter...
