If you felt uncomfortable then you were wise to move. That even goes for driving through a part of a city that you would feel uncomfortable.
In our 16 years of full-timing we boondocked on public lands throughout the country 90% of our time. The only time we felt uncomfortable and moved was when some yahoos set up their target practice and proceeded to fire away. We don't carry guns so just this noise made us uncomfortable.
I will note that we know of folks who have stayed in the area you mentioned and they didn't feel uncomfortable.