That is very low voltage.........BUT that is checking it with a bad battery still connected.
Disconnect battery cables then check converter DC output.
With 120V AC IN.....should have 13.2V DC OUT.
Voltage lower than 13.2V converter is bad.
YES you can use just battery power w/o converter and charge battery via a battery charger/generator.
Disconnect the 120V AC power source to converter (open circuit breaker/remove AC wire from CB if it is on a shared CB/unplug power cord if it uses one)
Mark battery cables prior to disconnecting them.....if connected backwards you will blow the 'reverse polarity' fuses on converter and will not get any power from battery to dc dist bars/panel---fuses
Also make sure new battery is fully charged PRIOR to connecting/using.
Otherwise you will damage that new battery right from the get go.