4-5VDC is usually what a 12VDC battery reads when it has a shorted cell or two. It could be the converter may have boiled out your battery fluids over time and it has shorted internally. Check the battery for proper fluids and if that is normal then check for blown fuses or circuit breaker between the battery terminals and the converter terminal. There is also two REVERSE POLARITY fuses on the 12VDC distribution panel that may blow if the battery has a shorted cell in it or maybe occured when the battery was sparked good when inproperly connecting the terminals in reverse order.
Like said above with just the converter plugged in too 120VAC and not connected to the battery it will put out a constant 13.6VDC.
The two areas of concern here is the converter/charger unit and the batteries including the connection path between the two units.
Roy Ken