We've been to both Fishermen's Memorial & Mystic KOA, both decent camping spots (preference to FMSP, though). Ditto on the suggestions to visit Mystic Seaport & the Submarine Museum!
I don't know if he's still a working judge, but if you get a chance to attend a court session in Providence where Judge Frank Caprio is presiding, do it. (Visitors are apparently allowed - or at least were before Covid.) He is remarkable, so much so that a TV show was made of segments of his court sessions. We've enjoyed the show.