No problems with that route. US64 between Angel Fire and Taos is actually the slower, more mountainous part of that route. But, it is a nice scenic drive. Also, you don't actually go through Taos. Taos is bad to drive through on the weekends during the summer, but otherwise OK. At one time they were going to build a bypass road.
If you are coming down from north of Raton, we would take US160 at Walsenburg. But we have taken the long way using I25 through Santa Fe to Los Alamos (similar to going to Espanola). Since it doesn't take much longer if at all. Although driving through Santa Fe is annoying also.
David