It's no different than the 12V fuse panels that have blown fuse lights in the fuse panel, just with the LED (and associated resistor) in the fuse rather than the panel. The fuses presumably would have a pair of LEDs back-to-back so as to be polarity insensitive.
An ordinary multimeter suffices to check fuses. You can either check for voltage across them when in place (there should be none) or check the resistance when they're pulled (it should be a short circuit).
I personally would not bother with these fuses because I have a fuse tester/puller in my RV all the time, and my fuse panel is inside rather than outside, and the fuses are reasonably well labeled so that it's not too much of a guessing game to figure out which one would be keeping the furnace from running.