Consider leasing a lot for the season next year. We have three campers - side-by-side seasonal lots on Smith Mountain Lake (VA) so friends and family can join us and a van camper so we can travel the state.
On a normal weekend during the season, I get home from work between 4:30 and 5:00 and we're on the road in 30 minutes and at our campsite in less than 2 hours and unpacked and set up in 10 minutes. We pack clothes and a cooler, that's it.
When I can take a long weekend to go some place special, we take the van.
Check out the campgrounds that offer seasonal sites within 2 hours of your house. You may find one with lots of kids and that feels right. Ask about the rules - you want to know that your style won't be cramped AND that other people won't be able to turn your season into a nightmare. Who knows, you may find something that gets you through the next couple of years and allow you to camp.