You need to do what's best for your family. In order to discuss it intelligently, I'd recommend you make two lists.
On the first list, put down the pros and cons of staying in your current situation.
On the second list, put down the pros and cons of moving into a RV full time.
Then, with these lists in hand, have a family meeting to determine your best course of action.
I do this with every major decision and it always helps us get properly focused.
Now, even though you say current zoning would permit the RV on site full time, have you considered what it would do to the property values of your neighbors? Are there other property owners doing the same thing?
Jim