Cape May NJ length of stay

So I am in the planning stages of our trip. we are staying in NJ (Liberty Harbor RV Park) to see NYC I`m planning on 2 1/2 days. is this too much, not enough? want to see Ellis Island and the Statue o...