I had an earlier version of the 3.5L in my 2008 Highlander, and I towed trailers in the 2700-3500 lb range for about 140,000 miles (yes, that many miles of towing, work and play). It held up well and performed decently. Towing up a long grade at higher elevation (Rockies), I liked to take it in 3rd gear at about 40-45 mph so the tranny wouldn't get too hot.
Keep in mind that 650 lbs hitch weight is not much. This will be your limiting factor. You should assume that a TT with 450 lb dry hitch will be about all you'd want, because once you add LP, battery, water and gear that tongue weight will climb fast. So you are going to be looking at trailers weighing probably less than 3500 lbs (dry weight) or so, and maybe no longer than 25' (if even that). I'd also recommend TTs no wider than 7.5' and not overly tall, to keep the frontal area as small as possible; frontal area is a bigger factor than weight when you tow on a typical highway, and weight becomes more critical going up and down steep grades. My TT was 7' wide and only 99" high; the other trailer for work was a 6x12 cargo.
I hope that helps.
Mike G.
Liberty is meaningless where the right to utter one's thoughts and opinions has ceased to exist. That, of all rights, is the dread of tyrants. --Frederick Douglass
photo: Yosemite Valley view from Taft Point