I'll second the park model suggestion. These can be found at many RV resorts, not just in FL and AZ. It makes sense if all you want to do is head south for the winter. You could then rent it out during the other months.
As for "how small is too small," that's a tough one to answer. Among other things, it depends on how old the kids are and what your family is comfortable with. There have been families of half a dozen full-timing in converted 15 passenger vans, but that is packing them in rather too tight for me.
While you can get trailers delivered by a third party, that's an expensive way to go; an option for a one-time move, but cost prohibitive for multiple moves.