We love Fort Wilderness, but it has gotten very expensive. The first time we visited the Fort as a family was 2005. We spent $38/night with AAA discount for a premium site in November. That same site 12 years later is over $100/night in November and they no longer provide AAA discount. We used to go twice a year, once in the spring and once in the fall. We would spend most of our time at the Fort. There's plenty to see and do and it's usually quiet as everyone else is at the parks. In the last 12 years, we've only done the parks 3 times.
Disney in the summer is not such a fun experience. Too hot and too crowded. We don't go from May through September. October to April is our favorite time, avoiding NJ Week (first full week of November), the holidays and spring break.
-Michael