Safe-T-Alert Power Supply

Does it maybe eventually turn off? You may have some capacitors scattered about the DC system that can power it for a relatively brief period of time. If that's the case, turning on e.g. an incandescent light or fan motor should make it turn off (and stay off) quickly.

Alternately, there may be some sort of a relatively high impedance sneak path around (or through) the disconnect solenoid, either through some device connected on either side (a voltmeter/battery monitor, for instance) or through some wiring oddity (conductive cruft of some sort on the solenoid, for instance).