When I go to NYC, I stay at Liberty Harbor RV Park in Jersey City, NJ. Right across from Manhattan You can get there on the commuter ferry (during the workweek only) or walk up Marin Blvd and catch the PATH train over to the MTA hub at 34th St. From there you transfer to the C train on the blue line and then transfer to the D train on the orange line and that takes you out to Yankee Stadium.
RV Park is about right. Its a place you park your RV on a patch of asphalt that has been fenced off from the huge commuter parking lot. It has water and eclectic hookups, a dump station and shower rooms that are shared by the boat owners in the marina.
There are some decent restaurants and bars in Jersey City in the "nice section". There is a supermarket right up the street. The ferry for the Statue and Elis Island is about a 10 minute drive away.
website