Water filter for drinking water only?

We often "dry camp" at a camp ground that has electricity but no water at the sites. We fill the water at the campground and the water is not the best. We use a ton of water bottles for anything we ...