You are correct in that as you have it drawn there is not a charging loop between the converter and inverter however, powering everything through the inverter is very inefficient and overly complex. You have losses inverting 12V to 120V and then back to 12V through the converter to power the 12V items.
A much better way is to leave the 12V distribution panel connected to the batteries and disconnect the 120V feed to the converter. Another plus is that in the event of an inverter failure all 12V items would still operate.
