I spent my career for many decades as a new car dealer. What you are describing is ridiculous unless it is a part of the written agreement involving the transaction. If not, and you have already taken delivery of the new unit and given the dealer the trade...you are done. You owe him nothing after the fact! He is the dealer and the expert here. It is his responsibility to inspect the trade and to evaluate it before the transaction is consummated. It was not something that was hidden and was in clear sight.
If your responsibility was listed in the buyers agreement originally, it is a whole different story and both parties made a clear mistake. The time for an agreement is before the papers are signed.