Actually, I have the same scenario as you do but with property in Northern Michigan (43 acres).
I considered building a cabin on it but quickly changed my mind because of the vandalism aspect. People will break into an unoccupied dwelling / trailer / whatever if no one is around because for the most part today people are not respectful of others property.
I solved my issue. I bought a truck camper for my pickup truck and we take the truck camper to our property and camp in it, pull our boat or quads and when we leave, we take it with us. Fully self contained and very portable plus we use it for travelling too.
Works very well for us and don't have to worry about trespassers or vandalism.
