I would shut it off, until it can be looked at.
If it does have a leak or bad capacitor, then running it is very bad for the expensive compressor. I am guessing that while on the roof, you did not feel really warm air leaving the unit - a indication that it was working right.
If you feel up to it, shut off the circuit breaker, and climb on the roof. Remove 4 bolts that hold down the outside cover. They should be large phillips screws.
Once that cover is off, look at the compressor wiring. It should be black compressor, with 3 wires going to it. One wire will go to a 2" diameter silver capacitor (or it could be oval). If this is popped up, then it is bad. If it looks 'normal' (yea like you might know what that looks like?) then it can still be bad, but you would need a capacitor tester to find out. . .
I bought a clamp on amp meter at Lowes Hardware for $40. It has a capacitor checker, and can measure amperage by clamping the ring around any AC wire, and setting it on the 40 amp or 400 amp setting. It even glows red when there is voltage over about 50 volts inside the wire (without touching the wire).
If the capacitor reads 28 mfd while the rating is 30 or 35, then replacing the capacitor will probably fix it.
Good luck,
Fred.