Airdaile's suggestion seems like one decent option to me. I've never taken the ferry myself, but it looks good on paper.
Another option that might be sensible would be to cross into New Jersey and take the GSP to I-87 N, through Albany (following I-87 rather than the Thruway). Take I-87 exit 20 at Lake George, follow NY 149 to US-4 and into Vermont, and then just over the Vermont border take VT 22A north to US 7 north. Partly it might depend on exactly where on Long Island you're starting from.
Taking I-95 to I-91 (or meandering over to I-84 and thence to I-91) is certainly doable but not much fun at all and can have pretty heavy traffic.