Just a few more bits of input for you:
-Design what you think your end goal will be upfront (Or at least, get close to it.)
-Source components that support that end goal. (Even if you start with smaller panels, battery size now - source charging and controller and cabling, etc. - that will support your end goal. Will keep you from possibly needing to replace a component later - buy it once.)
-Instead of turning off your fridge, consider placing a smaller inverter to power the fridge, into your design. (This way you can turn off the large inverter when not needed, and keep the smaller inverter with less overhead AH's usage running, to power the fridge. It also provides some redundancies for you, in case an inverter goes out - you can swap things around on the CB to feed power to the fridge from either inverter source.)
Best of luck to you, and have fun along the way,
Smitty