Do not test at the battery.
Test voltage at the converter's terminals or if can't get to them, use the battery lugs on the DC fuse panel near the converter.
You could have the battery disconnect switch or the battery fuse open and then you would see 12.x battery at the battery even if you could see 13.x at the converter or fuse panel.
So the converter is only tested bad if it is plugged in and you only see 12.x at the fuse panel, which means you are on battery with no converter (and it also means the path from the battery is not open anywhere)
Yes you can use a battery charger on the battery while connected. As long as the charger is putting more in than the rig things are taking out, the battery will gain in its recharge. So it is better to have a charger with some amps to it, and not use a trickle charger.
New battery? Possible oops there! Check the reverse polarity fuses on the converter if it turns out the converter is dead.