I went with a 30 amp system for much the same reason as you are. People are tending to go with 50 amp for a number of reasons - because it's available at many parks now, because they can, because it leaves options open for upgrades later, because it opens options for 240 volt appliances. For me, I had a 3000 watt inverter generator, I have a 4000 watt inverter/charger, I have 30 amp cords, so it was logical to go with 30 amp.
Your sketch shows a logical layout that I am going presume a few things about. First comment is the use of a simple plug option to switch between the shore power and the generator. It's functionally a manual transfer switch, and a better way to go would be a 30 amp automatic transfer switch. Why better? Simply because it can be automatic - shore power is on unless there is no shore power and the generator is on - and because cord plugs, particularly the big 30 amp ones, do wear out and have been known to cause arcing and over-heating when they get worn. Second, the "cam switch" is basically just a manual transfer switch again. It switches live and neutral from the plug (shore and generator) or the inverter. I am going to presume that it also can be set up so that your converter/charger can never be powered from the inverter, or you'll get the inverter trying to charge the batteries from the batteries, which doesn't work for long... I would use the Progressive Dynamics panel, I used one myself and it is a fairly decent charger with an AC breaker distribution panel and a legally separate DC fused distribution panel. A little light on AC breaker positions but you can get double breakers to increase the number of branch circuits.
My recommendation is to use automatic transfer switches wherever you can, but the manual option always works too. Always keep track of neutral and ground, and only bond neutral to ground at the source of the power, be that the shore power, the generator or the inverter. Chassis ground is always chassis ground and is common to everything.
Brian