Copper has resistance. Anytime you have current flowing there is a voltage drop because the copper is a resistor. If you add some dirty or loose connections that resistance is added and the voltage drop increases. If you have water flowing in and out of a tank at the same rate and step on the hose that drains it the Pressure/volume will build up. This is the voltage build up you are measuring and feeling. Dropping the jacks with no boards to insulate them will drop the voltage because this provided another ground return path.
