Looking at this only from an available vacation time and cost POV since you've already made it clear that your family enjoys the outdoors:).
Not sure if you own the boat you currently use during the summer but if you do you've a pretty good idea of what it costs to maintain and that's comparable to the commitment needed for an RV IMO.
When I decided to take up tent camping again I figured out the cost of all the equipment I'd need and then compared it to that of a different type of family vacation; took approx 75 days before I reached the break even point.
I know that the breakeven point for RVing will take considerably longer to reach with a usage time of only 2-3 weeks annually.
Try and do the same for yourselves and you'll better be able to make a decision I suspect.
HTH!
