In nearly 500 nights of camping out West in the last nine years, we've done lots of boondocking -- no trouble at all. Twice, in campgrounds, we had loud people near us: in a campground near Los Angeles, a bunch of young men played loud music all night -- they were so drunk that we did not ask them to stop. In another campground in the Sierra, a guy ran his generator all day and was rude when we asked him to move it away from our trailer. He complied, but he gave us his best frowny face.
Bottom line -- if you want something to worry about, how about all of the folks texting and driving?? That is deadly and is far more likely to ruin your day.
(I have a silly question, since I don't text -- what is so urgent that people have to text and drive? I just don't get it.)