OP, we stayed at Delaware National Seashore park in August. Its about as close to the ocean as you can get and still have full hookups. Its right on the Indian River Inlet, which cuts the park in two. Its a 2 minute walk to the sand and ocean. Fair warning... you have to cross Rt 1 to get to the ocean. You actually walk under the large bridge that goes over the river, so you aren't crossing a road... but you are parked right next to it. The views aren't the best towards the ocean. You are there for the convenience and full hookups for a reasonable price.
15 minutes north gets you to Rehoboth Beach, DE which is a family friendly beach and boardwalk with fun shops and a funky downtown area. 5 minutes south gets you to Bethany Beach, DE which is a much smaller family friendly beach town with a small boardwalk and some shops. Half hour south gets you to Ocean City, MD which is a large beach town with all kinds of stuff.
Another option, if you can do without electric, is Cape Henlopen State Park. This is literally just across the street from where the ferry lands in Deleware. Gorgeous state park, easy access to the beach (but not as close as Indian River), great hiker/biker trails, they have a small restored WWII Army base with an observation tower you can climb. Lots of beach, a parking lot and bath house at the beach. The RV parking spots are essentially parallel parking along a street with no hookups, but the park is beautiful.
Assateague has 2 campgrounds, one with electric, one without. I've never stayed at either, but I have visited friends at the one with no electric. It is MISERABLE there. No shade, no electric for AC, no views of the ocean (dunes), large flies... No thank you. Having electric might make things better, but in the summer you will basically spend your entire time inside in the AC... why bother going to the beach to sit inside your trailer??? As you can tell, I don't see the appeal of these campgrounds.