Not wanting to doubt the bad battery suggestions as it could be possible, they are original 6v x2 golf cart batteries, but why would they totally drop power instantly rather than slowly dim?
Also you were right, I got a better look at my multimeter touched to the battery terminals when we tried to start the generator again (which of course didn't even try) and the power instantly dropped to 9v, and even stayed there with the lights now flickering, then after about a minute the power instantly jumped back up to 12.3v and the lights went bright.
Guess I might not be used to batteries working that way when they get old, just thought it would slowly dim the lights.