I too would park the test light at this time. You need solid voltage information now. Could easily be a ground. Start right at your starter solenoid. If you have a smaller relay driving the starter solenoid you need to measure voltage on all four terminals or the metal housing if there isn't four terminals. Drag your shoreline cord around and use the ground lead so you will have the same reference while taking any measurements.
One trick on a sloppy fuse holder is to twist the fuse legs a bit.
With all of this my question is do you hear any clunking near the engine area every time you turn the key or only when it decides to engage? If you hear any solenoids clunking and no engine cranking that is where I would start with voltage measurements.
