I would do it, but we still have 2 young kids, ages 9 and 6. Both myself and DH have retired, me from law enforcement and he from teaching. We still want to have a S&B home for the girls, but we do part-time full-time. We live in our 5'er from the last day of school in May until Labor Day in September. We will get in 100+ days this year.
I would sell out of the big house, get rid of the extra furniture and either live in the RV you have now or rent a small apartment and save up for the nicer RV when you are ready to go full time. In 7 years, your current RV may or may not serve you well. Just something to think about, along with everything else.