If you were okay pulling a 34' (total) trailer, the extra 3' probably won't matter. I would imagine going up from that would be a pretty big jump though. We have a 33' (total) trailer, and I am comfortable with it, but it is bigger than I want it to be.
As far as I'm concerned, the trailer is for sleeping and hiding from the rain. To that end, we can easily fit all three kids (8,10,17) and a few friends. We spend 90% of our time awake outside, and very rarely let the kids play inside if it's not raining. The point of camping (for us) is to be outside, without having to sleep on the ground.
If I could convince the DW, I would go smaller, but such is life.