It depends upon who the RV dealership uses for financing. Some companies like Bank of the West fully understand the RV lifestyle and full-timers. They don't hold using a mailing service as a residence against you.
Some others do consider you not being a 'home owner' a mark against you.
Quite frankly, I'd suggest you call a couple RV finance companies - those who advertise as financing RVs. See what their rates and terms are. Compare them to what the dealership offers. You should get a better deal if you have a couple companies offering you a loan.