Does the battery voltage recover immediately when the inverter is turned off? If so, it may be a relatively high resistance spot in the circuit and not a battery problem. This may be a poor connection at a battery terminal, for instance, caused by corrosion or crud on the wire terminal, or a poor ground, or....
I have a somewhat similar problem on my battery, which I have tracked down to be the connection on the battery (in this case a "marine/RV" pseudo-deep-cycle battery) between the screw terminal and the large round stud--that is, on the battery terminal itself. You can find these sorts of things by measuring voltage drops along the circuit while its under load--or, if bad enough, by looking (carefully) for warm or hot connections.
Of course, the problem may be an actual battery problem and not a connection problem equally as well.