A handful of attraction suggestions that I know of:
DC -- So so many museums to see, many free of charge. It's not quite coastal, I know. I stayed at Greenbelt NP when I was there and found it very convenient and pleasant and inexpensive, but some may not like the lack of hookups. Cherry Hill is not far from Greenbelt and offers a more RV park/resort atmosphere (and, by most accounts, does so successfully) with prices to match.
Philly -- Not coastal, but a lot to see in the city, including historical sites and the fabulous Art Institute.
NYC -- Of course there is a lot going on in New York. I can't give too many specific suggestions. Liberty Harbor is the usual suggestion for a nearby campground, notable for its location but not at all for its amenities.
CT -- Mystic Seaport
MA -- Boston has a lot to see, museums and historical sites and a lot of pretty parts of the city generally. Avoid driving through the city (though driving under it--without the RV as propane is not permitted in the tunnels--isn't so bad). Parking and navigating in Boston is hard, even by large city standards.
Cape Cod is arguably the traditional (southern) New England seaside vacation destination.
ME -- Acadia NP is a must-see.
If you go inland some, New England and the Adirondacks and finger lakes regions of New York, among other areas, are beautiful. I may be slightly biased in my opinion, I'll readily admit.
