RV site close to NYC?

YOU can stay at a hotel in Secaucus with good access to NYC via NJ Transit bus to Port Authority. Trip less than 45 minutes in rush hour, 20 minutes during other times. They have free parking. Check Clarion hotel Secaucus and others including Extended stay America (check reviews on this one).
Also some county parks on Long Island have camping

Looks like there are a few on Long Island, but we have never been to any of them. Take a look at RVPark Reviews and check their NY map.
We have used the KOA mentioned above. It fits your needs, other than the fact it is quite a ride to the city.

Liberty Harbor is the only place to stay if you are going to NYC to visit NYC! If you are going to "CAMP", then don't plan on Liberty Harbor It isn't a mile walk to the PATH train, more like 1/4 mile. My 5 and 8 year old was able to walk it with no problem and we never felt unsafe, even returning after midnight.

You are right, Liberty Harbor Marina and RV Park, Jersey City, NJ, is just a large gravel parking lot with water and electric hookups. It is not a place you can sit around your campfire and visit with relatives. But, if you are there to visit New York City it is in a great location. It is just a mile or so walk to a train which takes you under the Hudson River to the World Trade Center in Manhattan where you can change to subways to get to anywhere in the city. A commuter ferry also makes a stop right at the park.

The only other CLOSE Campground would be the Newburg KOA . about an hour ride WITHOUT traffic...