You moved into a situation that had already been established. That's hard.
You are related to a selfish inconsiderate person by marriage. Unless the marriage ends, or one of the 3 of you die, you have to consider all involved before you can make any changes (that includes the grown kids). That's almost impossible.
It's really easy for people to say "throw him out" - they don't have to deal with the aftermath of the action. You have to consider your relationship with your wife, your kids, etc. Throwing him out could create a bigger problem than having to listen to him spend your money.
Vent when and where you can. And remember, you've only dealt with his selfishness for about 30 years, your wife has dealt with it all her life.