It is not a difficult drive at all, as Colorado roads/passes in the mountains go. There's actually 2 passes along the CO/NM border, Cumbres Pass and La Manga Pass. These passes are open all season, CDOT does a good job keeping them open and maintained, should not be any significant roadway surface/pavement issues to speak of.
Great photo ops for the Cumbres and Toltec RR trains at the top of Cumbres pass if you time it right. Once over both passes there's a drop down to a valley and then valley driving all the way in to Antonito, a really beautiful drive, going from mountains to high desert. Antonito to Alamosa via US 285 is flat with great views of the San Luis Valley. From Alamosa to Walsenburg via US 160, you have N La Veta Pass, which is also not a difficult pass, and RVers and truckers use it all the time.