Knowing sweet F.A. about the details of it, but....
If you have room for a 2nd batt in the exterior compartment, what about a) insulating it, if there's room inside the compartment with the batt's or surrounding the outside of it, if that's possible. If do-able but not enough on it's own, could a small duct be tee'd off your existing duct work to feed into the compartment.
Or b) Using battery warming blankets/pads? Uses power, but likely little and maybe be easyier and cheap then the reno you're considering. And still add some insulation to the compartment. Or at least an inch piece glued to the inside of the door and maybe a better gasket around the opening to keep most of the cold out.
