It's certainly doable but there are trade offs with any choice in life.
One thing I would question a bit is are you planning to take the rig on these weekly 3-6hr jaunts to family? Fuel bills could get pricey real quick at 10mpg.
I wouldn't worry too much about the "starter unit" idea. Assuming you are reasonably handy, most stuff is fairly straight forward to maintain on an RV and I wouldn't count on a high end unit holding up significantly better the kids are rough on it.
I would suggest getting an older inexpensive unit to try it out with the idea that you will trade out in a couple years once you know what you really want. With no RV experience, it's easy to get sucked in by slick marketing of features that really offer little or no benefit. Plus if you buy an expensive unit, it's a lot easier to lose a lot of money if you decide you don't like it.
You should be able to find something used in the $12-15k range and as long as you don't destroy it, you should get most of your money back out. Big thing to watch for is water damage. If you aren't familiar, once you've got a rig you like, paying an RV tech a couple hundred to give it a once over is likely well spent money (even new rigs it's not a bad idea as there are often teething pains on new units).