I've never removed old caulk from my roof, I've just added on top of it, and only in the places where I've seen any cracking (which has been very few over many years of RV ownership). If you pull it off, you run the risk of damaging the rubber under it, by pealing, scratching, rubbing, pulling, the old off. I find it best to just leave the old. Wash it real good and let it dry good, then add more right on top of the cracks.
You want Dicor caulking, made for rubber roof's. Anything else could cause a caustic reaction over time and actually deteriorate the rubber. Thus, another reason to apply right on top of the old.
Good luck.