Your fridge needs heat to circulate the fluid/gas that extracts the heat from the fridge and freezer areas. If the area in back of the fridge is too cold, the fluid/gas can't circulate. Sometimes all that's needed in colder weather is to temporarily cover the most of the vents in the outside access panel to slow the flow of cold outside air up through the roof/upper vent. In even colder weather, a 60 watt incandescent light bulb (if you can still find one) may help warm up the area. Make sure it's in a safe holder of course, such as a mechanics lead light.
