Our old 1990 Winnebago Warrior has the generator directly behind the driver seat. It's under one of the kitchen table seats. So unless you were sleeping at the kitchen table bed (no one ever does), you would not be over it. It's definitely not bad when you are in the back bedroom. It's a pleasant white noise roar.
But it is a noisy old beast. We do a lot of boon docking but typically we don't run the generator at night. Even in the south it usually cools off enough in the night to be OK for sleeping.
I definitely recommend installing CO detectors in your RV. I installed two - one in each sleeping area. One in the cab-over bed, and one back in our bedroom. With a 25-year old RV you never know when a furnace or generator or whatever is going to malfunction and start leaking exhaust gas inside. Obviously a propane detector is essential also. Most new RVs come with them but I had to install one in ours.
I had the generator running when I had the RV at a shooting range one afternoon with the kids. After a couple of hours one of the CO detectors went off. Clearly some exhaust made its way inside - probably through the open door (the generator was on to charge/run the kids' ipads).
If you are going to be sleeping with any kind of flame appliance or generator running CO and propane detectors are essential.
A propane fired furnace still needs electric to run the fan, it will run on the battery ,but will kill a single battery overnight, if you set at much above 60, it may even drain 2 batteries, depending on state of charge when you start.
My RV came with a small house battery under the hood. The furnace would run down the battery overnight. In installed a second huge deep-cycle battery in one of the storage compartments (I vented it) and now I don't have problems with the furnace for a weekend.