Nick
Depends on where you're camping. If it's in an organized campground you really have no right to expect more than them being on their own site. If you're camping in a disburssed area then "out of sight and sound" would be reasonable.
Yes I agree you have no expectation of being alone in a campground but if there are 200 sites and there are two people camping in them why would someone move in on top of you?? Just no real reason for it.