If the switch is identical to the light switch, it's a switch that originally ran to the exhaust fan. Although it has wires attached to the switch, they were probably disconnected at the fan, and the fan was wired in direct, thus using the switch on the fan, not the wall any more. It's possible the wall switch failed, and a replacement could not be found that match. So it was just wired around the switch.
Either that, OR... the switch has failed in the "ON" position. No matter which way you flip the switch, it's "ON", thus, needing to use the switch on the fan to turn it off. Disconnect one of the wires from the switch and see if the exhaust fan is disabled.