Battery Problem

I have 2 6V golf cart batteries in my 5er. Batteries are old and due to be replaced. When I checked the water level, the level was a little low but it was obvious one of the batteries has cooked the water and overflowed. Can I assume this simply means the batteries need replaced? Or could it mean the converter is faulty? Electrical is not my strong part. Any help would be appreciated.