As someone else mentioned, if not Liberty Harbor than you'd have to come toward my neck of the woods about an hour West. There are a couple of campgrounds that are 10-15 minute rides to bus and train station. If you do go that route, the train to Hoboken is best (about an hour on the train). The PATH will take you from there directly to Ground Zero (11-12 minutes if memory serves).
That PATH station still creeps me out. I used to take it every day to the basement of the Twin Towers. Now it's pretty much open to the sky above - it's surreal.