It might sound like we are all telling you not to do it but more just thinking of things you might not have considered. I would love to go full time but DH isn't interested.
Saying that, if you have any equity in the house you only have a certain amount of time to reinvest it without getting a tax kick in the butt. You might want to use that time period to try out RV living and reassess then. You can always buy a small condo.
I don't blame you for wanting to ditch a house you barely use. I look at these monster homes that people purchase and wonder who wants to clean all those windows.