A reasonable filter, (I use a very simple charcoal filter) can often fix "Funny taste" and there are some filters out there which make my filter look like a piece of pipe (I'm saying they are way better), Some of the multi-stage carbon block filters can dang near distill it (Ok so I exaggerate slightly).
A Brita pitcher is very good at filtering what you drink.
Now I can see bottled water for some things... Mostly where a source of water is not easily found.. But when you have 50+ gallons in your fresh water tank, Or city water.. WHY.
About the only thing I have found my filters will not handle is the water where I park in the winter.. The city, I do believe, over chlorinates it. Cause it tastes a bit like bleach, even after filtering... THE SOLUTION...
Fill the on-board supply tank, let it sit even just a few minutes (Takes about 15 to fill the tank so by the time I get inside the water in the bottom has been sitting there at least 10,, I think the book says five is needed, (The book is on keeping pet fish alive by the way).. And by the time I pump it out.. It has dissipated.