I liked the idea of using the existing conduit and upgrading the wire size and installing a sub panel until you mentioned a non-conforming garage and someone suggested solar.
It doesn't seem like you are interested in building a new garage closer or attached to your home, so I would consider pulling some #10s and a disconnect at the garage and CB in the main panel. Then as I could, I'd purchase 300 or 400 watts of solar panels and controller, add batteries as needed.
You could put the panels either on the garage or the RV. Unless someone was going to be sleeping in the RV, run off the solar. When someone is using the RV, the new wires would carry at least the AC or the MW and the batteries could carry the house 12 volt system.
A full upgrade will never give you a return, the solar will, especially if you can add the batteries to the RV and you want to dry camp.
