I normally don't think that it is necessary to remove the TV from it's mounting, but in this case that is about the simplest solution for the question. In my case, I'd find out what the side of the fridge cabinet is made of and back as necessary and mount it on a swivel mount on the fridge cabinet. I'd find one with a reticulating arm and use that.
Is it articulating or reticulating?, I can't spell and don't have a very good command of the English language, sorry. No dictionary pop up on the forums.