As a matter of principal as much as anything, I prefer to avoid toll roads if reasonably practical. Here in the northeast it is not always reasonably practical.
EZPass is a debit system, but in some cases the discounts available vs. cash will more than make up for any amount you have left in the account and any initial setup fees even in a single trip. If you have a motorhome and are going on I-90 across New York, for instance, the motorhome discount plan will save you a good chunk of money (about half the toll vs. cash) if you request it with a New York state EZPass transciever. The different states and agencies in the EZPass consortium have somewhat different fee structures so it's worth checking a few out for the best deal: things like initial charges, monthly fees, etc.
Paying by cash or by mail is certianly doable. Paying by credit/debit card rather than cash on roads that are not toll by mail is not always an available option.