When you're plugged in, the little battery voltage meter (I assume you're referring to one with three or four little lights on a tank level/information panel sort of setup, but it's true for any voltmeter) will show the charge voltage, not the battery resting voltage, since that's the voltage being applied. For little lights, this equates to "fully charged" practically all the time.
If you're starting the generator while still plugged into shore power, you are getting a sizable portion of the power to crank the generator from the converter rather than the batteries. My Onan 4K requires somewhere around 100A to crank, so a 40A converter can supply almost half that and make for a noticeably less severe load on the batteries. That's not very useful when you actually need to run the generator (i.e. when you don't have shore power available).
Charging the batteries fully or nearly fully from the converter when they're quite discharged might require a day or so...for sure more than a couple hours. As others have said, it's not unlikely that the batteries need replacing. Lead acid batteries do not take at all kindly to sitting in a discharged state.