Unless you're really running the refrigerator on propane and only using the 12v for the control board, your setup won't work. The refrigerator running only on 12v for cooling will drain your batteries in a few hours or less.
If you are running on propane then your setup will probably work as long as you've got good sun for a decent amount of time striking the solar panel almost directly head-on. But unless you can recharge the batteries to full charge state every day you'll probably notice less and less battery power after a few days. (Unless you're using Lithium-Ion batteries.)
My previous RV was a Class B and I had a portable 120W solar panel. I used 12v only for the refrigerator control board, minimal lighting, and to run the furnace fan for a few minutes on cold mornings. With pretty good sun for long hours I found I could go about 3 or 4 days before the battery needed to be really fully recharged as the solar just wasn't getting me there. My batteries were deep cycle 6v flooded.