I wouldn't worry about it as they must of knew what to expect from a family campground, particularly on a busy holiday weekend. Glad to hear you and your kids had a great time even with unfriendly neighbors.
There are many campgrounds that cater to virtually any type of camping anyone could prefer, read over campground rules and regulations to see what is or isn't allowed before plopping down your money.
After that you could only blame yourself for your choice, as long as they aren't breaking any rules or laws there's not much anyone can do but move on or grin and bear it. Doing some research beforehand helps avoid these situations.