It's not always a matter of privacy, but also a significant matter of safety and liability. If the parks allow RC flight, then in the hands of a good lawyer, they would share at least part of the financial responsibility if somebody got hurt or killed.
When I use to fly, all the regulated parks were well outside of town, and you were required to carry liability insurance if you wanted to use their facilities.
I have a 75lb compound bow that shoots an arrow in excess of 300 FPS. I hit my target....most of the time. I don't think a fellow camper would be too happy with me if I was shooting at my target bag that backed up to his campsite