What kind of WH system do you have..........1 valve, 2 valve or 3 valve?
1 valve.......3 way valve that controls water to WH cold inlet. Turn it 90 degrees it 'bypasses' WH cold inlet, ties into WH Hot outlet line and a check valve in Hot outlet stops water from backflowing into WH.
If so....BAD CHECK VALVE
2 valve.......both valves have to be set to 'bypass' position otherwise flow to WH is not stopped
3 valve.....cold inlet and hot outlet valves have to be CLOSED and 'bypass' valve as to be OPENED
Improper valving OR bad check valve.
Probably bad check valve.........common problem.
Remove check valve and install another valve.
Here is another post....same problem.....with pictures.....
LINK