Fridge temperature depends on ambient

Let's say I have our (Norcold) fridge temperature set on level 5 and the temperature is holding steady. But if the outside temperature goes up or down the fridge temperature does too. If it goes up, I...