In my mind, the further away from a big city that you are, the safer you will be. We have had a very few instances of loud obnoxious types camping near by and every instance was in a developed CG. All in the south.
The one time I felt a bit concerned was when we drove thru a undeveloped campground full of the 'Rainbow Tribe'. That was in the Ocala NF and we weren't looking to camp there. But they were unpleasant company. I haven't heard anything about them in the past few years, hopefully the tribe is extinct.
That being said, the Founders gave us the 2nd Amendment for good reasons. Use it.
BK