It goes more than skin deep. Almost all aluminum skinned TTs are framed construction. Wall framing built up from floor, skin applied to outside, wall filled with insulation, paneling applied to inside. Wall strength derives from the framing. Construction is labor intensive, material and captal costs low.
Many fiberglass skinned trailers are made with laminated wall panels, using a metal (usually aluminum) stiffening frame. Plastic outer panel, plastic or luan underpanel, foam core, and (usually luan) inner panel laminated as one piece. Wall strength derives from bond of inner and outer skins to the core material. Not a bad thing, some aircraft parts are now made this way, but delamination can be an issue. Construction uses much less labor, more expensive materials, and either more time (for vacuum bonding) or more capital (for high speed presses). Winnebago has been building tjis way for more than 40 years.
Third possibility is fiberglass skin applied to a fully framed wall. This is premium construction, used by Newmar, New Horizons, Carriage, Northwood (in the case of their fiberglass skinned models). It has high labor costs and high materials costs, so if you are asking the aluminum vs question ypu are not likely looking into that price range. Then there all aluminum models like Airstream, ans molded fiberglass models like Bigfoot, in yet another price/cost category.
Framed wall trailers, aluminum or fiberglass skin, can be easier to fix. Laminated wall trailers, it depends on the scale of damage. Little things, you might patch over. Major damage, you don't repair the wall, you replace it. For low cost, mass production TTs, that means, for insurance purposes, major damage to a wall panel is a total loss.