I haven't stayed at either of these, but have done a bit of online looking for possible future trips, so take this with a grain or two of salt.
Liberty Harbor has a great convenient location and some hookups, but not much else. It's perhaps the best option generally speaking.
Floyd Bennet Field (part of the Gateway National Recreation Area) has some RV sites--really more like parking spaces, I think--that are quite affordable. Unfortunately, getting there would mean driving through the Bronx on surface streets since the parkways are not usable by RVs, and it's not nearly as convenient to public transportation overall.