Here's a couple of thoughts. First, once you start living in the RV fulltime, you may not feel the same way about it. Before, it was a getaway, now it will be there all the time. You may want to try putting it in a park for 2 months and living in it fulltime before making any drastic decisions.
Second, I got out of a high stress job 7 yrs ago. Got rid of the house and most of the belongings. Is life all peace and tranquility now? Heck no! Is it better than before. IMHO, yes. But that's just me.
You may be able to realize your retirement much sooner without all the overhead of a house, not to mention your gains from the sale of the house and all the yard sales.
Good luck in your decision.