It doesn't really bother me per se, but I did grow up a boy scout and one of the first things you learn as a boy scout is that you have to ask permission before entering another person's camp site. Most of my experience has been with kids walking through the camp site. They're kids, no big deal. Most people's kids are brats and don't know any better so I don't get too worked up over it. Adults should know better, but the way I see it is if my biggest concern in life is who is walking through my camp site, I'm probably doing pretty well. :)
People letting dogs loose bothers me much more.