Three options come to my mind, all more or less doable, but with quite different advantages and disadvantages.
Liberty Harbor, as mentioned, is for many people the most convenient and probably best option. It's not a fancy campground (think parking lot with utilities), but it is at a very handy location with easy access to mass transit. It's just across the water in New Jersey.
Croton Point county park (Westchester county, north of the city) is very close to a commuter rail station and thus fairly convenient to the city.
Finally, I've stayed at Floyd Bennett Field, part of the Gateway National Recreation Area, in the Bronx on Jamacia bay. It's about a mile walk (or drive) from the campground across the park to a bus stop, and thence to the subway, maybe an hour to an hour and a half all told to central Manhattan. There are no hookups, though there is a dump station and potable water and shower house available. The RV "sites" are just (large) marked-off areas at the perimiter of a big concrete expanse, a former seaplane base/tie-down area. Getting there is something of an adventure as the obvious route is via parkways that RVs are not allowed on; the best route, from the north at least, is probably to take the expressways to Flatbush Avenue clear across the Bronx.
You do not want to drive (and park) in New York City in lieu of taking tour busses or the subway. To see all the sites, plan on staying for a long time.