We have stayed at the small state park in Columbus NH a couple of times. The small museum there is interesting.
Two observations:
One morning another camper told us a man had come to his door overnight asking for water, odd since this is a campground - there are water spigots every few feet. He had called HSA, worried.
From there we drove west along the border for 50 miles or so. Every few minutes we would see a border security car alongside the road or at an intersection, each giving us a wave as we drove by. I rarely saw another car. I became convinced that if we didn't pass by the next Homeland Security car on time they'd have come to look for us.
The people in Columbus regularly go across the border to shop or see a dentist. A fellow in the museum assured us we could walk over and back without worrying about a passport; we did not test that.
We have enjoyed our stays there. I would turn my phone off to avoid Mexican roaming charges though.