I don't consider it anti-customer in the least. It's a wise business decision-why should those costs be passed to other customers when YOU have other options? Most people carry multiple credit cards and I don't think I know anyone who would just walk away instead of using their VISA or M/C. Those that do IMHO are somewhat selfish and may be problem customers in the long run.
In multiple businesses I have had over the years. I have never accepted AMEX because of their high merchants fees.
With literally thousands of transactions I never had a customer walk away because of it. They always offered an alternative payment method without a problem.