It's not your ignition switch as the one at the back bypasses it. Take a hammer and hit the starter several times and then try although it sounds like a starter solenoid problem. I would take a screwdriver and short between the small wire and the large + lug on the solenoid bypassing everything other than the batteries and their cables.
*Having thought about this a little more, did you have the ignition switch on when you tried to start from the back? The ignition switch has to be on/working to give power to the injection pump. If shorting the starter solenoid makes the starter work, then it's back to the ignition switch.