This is one problem I have not encountered. I suppose it's my dogs that keep people out. Or maybe I pick good sites.
The closest I have come to this is at my favorite campground and site. The site sits just above a nice deep section of the creek and folks enjoy walking down to it. The people never walk through my site, but their unleashed dogs often wander up into my site. I just politely yell out "could you call your dog back" and they call the dog and walk away (and rarely wander back while I'm there).