Matt, you have gotten some good response to your question. We went from a 21ft to a 25 foot and it made NO difference when it came to driving and negotiating the challenges of getting around. It is a fact, in my opinion, that the longer the trailer the more limited your choices will be. If you are "off the grid" folks than you will likely use state parks more than KOAs. Most state and federal parks are set up to accommodate a 25 footer. Anything larger will start to limit you.
People will post that they have never had a problem finding a place to park their 32 footer. Well, once pressed you will probably find that they stay at private RV parks. We had a tent trailer for 17 years and there was not a spot I couldn't get into. Moved up to the 21 and did find a very few that wouldn't work. Now with the 25 I can still get into almost all sites but anything larger would create some additional issues.
The key here is choosing a trailer that is right for you. I certainly would not rule out a 25. If you do encounter restriction they will be few in number.