Mystic Seaport isn't too far away. It's in Connecticut. It's a restored whaling village, and the last time I was there (about 15 years ago) it was excellent. History.
When we were camped at Fisherman's Memorial we stumbled into an opportunity to get on a ferry boat tour to Block Island. It was a fund-raiser or something, and it was a half-day cruise that included a meal and some time on the island, as well as a complete boat ride to and around the island and then back to port. Lots of fun.
Of course Newport, but you already mentioned that.
-Speak