We only have 2, and we also camp mostly off grid. And I'm pretty satisfied with that. I make it a religion to charge them every day. Setting up the generator is just part of the routine. Whether you have 2 or 4 batteries, going 3 or 4 days without charging is not good for the batteries.
Having said that, I would still recommend going with 4 batteries, if you don't have an issue with the space or weight or cost. More battery is simply better, end of story.
One other nice benefit to having 4, is that if you have one battery fail, and they all fail eventually, you can drop down to a bank of 2 and keep on camping. Whereas with us only have 2, if one fails we have a problem that has to be solved ASAP.
Either way, make charging a daily routine. If your schedule away from camp forces you to skip a day, not a big deal. We like to time our hour of charging around something that uses power. Usually that is at shower time or when washing dishes. The pump draws a lot of power, and we get a better shower when the pump is running on full converter output.