Polyurethane sealants in general have far greater adhesion than silicone. If you look at a company that makes both (like 3M) they list the adhesion to different substrates of all of their sealants.
I think those in the industry like silicone because it is easier to handle and clean up than a good, sticky polyurethane. Also quicker curing than most good polyurethanes. On the RVs I have owned, they must have bought silicone by the boxcar load, because it was used in copious quantities, inappropriately. And in many cases did nothing to seal out moisture.
Polyurethanes can discolor in the sun, that is why I use 3M 4000UV for exterior. It does not yellow - the UV is for UV resistant.
If polyurethane must be removed, there are products that will do that. "Antibond" is one that seems to work pretty well (you can get it at marine stores like West Marine). I have tried a couple of others that don't seem to work as well. Another trick with polyurethane is that it will creep: apply some force (for example with a wedge) then wait a couple of hours, tap it in further, wait, etc. With patience, you can separate it.
Finally, many (most?) sealant joints are improperly designed, whether you are using silicone or polyurethane or butyl. You need some thickness of sealant to allow the two parts to move relative to each other, especially when they are different materials, like aluminum and plastic. These will expand at different rates, and flex at different rates. If you squeeze the joint so that there is a thin bond line, there is not enough flexibility in the sealant to accommodate the motion. Lets say you have a sealant capable of 100% stretch before failure, and a joint of 1/16". The parts can only move 1/16 before the sealant fails. If you instead have a 3/16" joint, now it can move 3 times as far before failure. Many times this can be done by inserting small spacers inside the joint to guarantee the sealant thickness.