In Vermont (and I think New England generally), zoning regulations are determined on a town by town basis. I think there are some areas where that would not be prohibited, particularly on sufficiently large lots.
Septic may be the biggest hurdle from a legal point of view if it's required. At least in Vermont, it's very easy for a septic system design and approval to be five figures--and that's before any parts are bought or dirt is moved. Electric service is, of course, also not inexpensive to install, particularly if fairly distant from the power lines.
DuchmenSport's approach may be more practical. The obvious downside is that you have a whole extra house to maintain.
