Yes, OP, it is possible. Evidently you have exactly the rig upon which it happens, whatever that is.
If you are towing a trailer, for example, and have a high capacity inverter, you need to know that your tow vehicle probably will only send 8-10 amps to the house batteries for a number of technical reasons. So, yes it is charging the house batteries, but not very vigorously.
If you are using a large inverter, say a 2000 watt one, it is going to draw a lot more than 8-10 amps just being turned on, let alone powering any load. So, yes, you can easily deplete the house battery while towing, especially if your house battery isn't fully charged to begin with.
And indeed, it is also possible that the alternator isn't charging the house batteries, either by design or because there's a problem.