You'll have to pay a toll to cross the Hudson unless you go all the way up to Albany, then it's the Mass Pike which has tolls. Also, I-95 in NH and ME has tolls.
Going to Hyde park, it makes sense to cross the Newburgh Bridge which I believe is around $15 in tolls. EZPass is your friend in the northeast. The Mass Pike is all electronic tolls.
From Hyde Park it is possible and quite scenic to avoid tolls, just not very fast. US9 to I-90 to Albany, I-787 to Troy then Route 7 in NY to Route 9 in VT to US202 through NH and ME.
Stay off the Taconic Parkway.