We do a lot of dirt road towing at low speeds. (See the links in my signature for more info on the trailer and on the kinds of places we have gone.) Our first trailer, which had low ground clearance, lasted us six years, and then we sold it for a pretty good price. It did not deteriorate due to the rough towing.
We have had our current trailer for about five years. It has great ground clearance and has had no trouble with rough roads. Honestly, trailers take a worse beating on the bad freeways in Calif than they do at ten mph on a dirt road.
Although I have had two Fun Finders, I do not think they are any better built than any other brand. I think that a well-equipped Casita or any other fiberglass egg will hold up better than a square trailer like mine, but I needed more headroom.
Also, I am sure that you are aware that the shorter the trailer, the more maneuverable it is (especially when boondocking). But a bigger trailer is (of course) more comfortable, especially in rainy weather. I hear it rains sometimes in Oregon. That sure sounds good to a drought-stricken Southern Californian.