Context means a lot.
From the supreme court response, looks like a big difference between being pulled over vs in a campground hooked up to utilities.
Not that you should give up your rights but if you aren't doing something to make your situation stand out, I don't see why they would ever ask in the first place. If you've been living the past 3 months on a suburb street, the neighbors probably want you to move on and he's looking for a reason to move you on.
A simple response would be to politely ask what his concern is?
- If they are looking for a kidnapped child, I will likely let him take a quick look around.
- If he can't come up with a good reason, I'm probably going to politely decline his request.