Discharged batteries sulfate and your's may have limited capacity left. Load testing can determine their condition. Find a good battery shop that will test in the rig for free. Weak batteries often cause other problems that may not seem to be related to the batteries. Even if your batteries are usable for your use it could be useful to know that they have lost say x% of their capacity.
The salesman switch turns off most loads and should be turned off when leaving the rig for longer times. Mine does not turn off the non res refer which will continue to run on propane. Your's should not disconnect the charger.
You should consider installing a 30A or 50A RV plug.
My rig is stored under cover so the solar doesn't help and I have no AC. So I installed a full disconnect switch on each bank so that the only drain is battery self discharge.