I haven't done it, but I've seen it a few times, usually a first or second generation Voyager or Caravan (not the long wheelbase "Grand") or the first generation Odyssey (which is still the size of Honda's minivan for markets in Europe and Asia). Front-drive Mazda MPVs and later Mazda 5 are a similar size, and can be dolly towed. With manual transmissions they could be towed 4-down.
Minivans got really big as more manufacturers came into the business with "hey, our minivan is roomier." Roomier is what the market wanted, large families carry a lot of stuff.
For smaller minivans today, look at the Mazda 5, Ford's Transit Connect or Nissan's NV200, although that may not be available retail as a passenger van, it is for the taxi market. Chevy has come into the market with a City Express, but I don't think they are doing a passenger van.
All of these smaller minivans share the compromise involved with the original Voyager/Caravan: you carry people or cargo, not both. Five passengers with huge luggage space, almost twice as much as a mid-size SUV, or 6-7 passengers and space for luggage of 2-3 people, the way Americans pack for travel.
A lot will depend on how many seats you need. NV200 is only five seats, Mazda 5 (now extinct) was comfortable at six, Transit Connect will carry seven but the rear seat is really cramped.