How do you feel about people walking through your campsite?

My wife gets upset over it, I usually don't. We commonly camp along a river and people like to walk along the bank, I'd be bothered if they stopped and put chairs in from of my view though. At another location we park close to a playground for the kids and sometimes expect other kids to run through. What bothers me is when kids we don't know stop and hang out with our kids and do not leave. They may be nice but I would not want my kids at someones site unless we spoke to a parent and they were ok with it.