Welcome!
I'm still pretty new as well. One season under our belt and chomping at the bit for spring!
In a lot of ways you are In a perfect position. No tow vehicle, no baggage of some older camper to trade in...
Go visit a lot of campers at dealers and Rv shows if you can. Even if you are going to buy used.
Go look at the ones that Interest you floorplan wise, spend a lot of time figuring out how it will work for you and family. Will the kids still fit in their bunks as they grow? Sit on the couch, and even the toilet and imagine going through some routines like cooking (can others still see the TV on a rainy day?)
Then once you've found "the one" find a towing vehicle that can haul it safely. This takes homework. Not all trucks are built the same and unless you've loaded it up for camping and taken it to the scales yourself the truck and trailer will almost certainly weigh quite a bit more than their stickers say they do.
One thing NOT to do is settle for a camper that is too small, you'll end up not liking it, or it won't function for you. You'll either end up not using it or trading it in and (if bought new) take a huge hit on depreciation. Take your time and do all the homework and try to buy your third camper first IF you are seriously committed to camping/RVing. It's a multi year commitment and I'm not trying to scare you away at all. Just make sure you are going in with eyes open.
My First ever camper is 36ft ball to bumper. There are plenty of folks out there that will say I was crazy for buying a brand new camper (financed even) that's "so huge" for just a few weekend trips. You know what. It's your money. If you can afford it and actually have thought it all out. Go for it! I can't speak for the West. Here in the East/Midwest I've found a few campgrounds I can not get my monster into. Some national park campgrounds will not work for me... But so far. Every time I picked a certain place to go I have always been able to find a site to fit my rig. Mostly it takes a little more planning and maybe reservations. If you prefer to live totally on the spur of the moment then the smaller campers will give you more chances - as you've already seen for yourself.
So. Don't let anybody here tell you "you shouldn't" or "you don't need". You decide that, do the homework, get the right TV, and get camping. The kids are growing like weeds... That's the only scary part - how fast they grow.