I had the same thing happen to my wife and I back in the days of flip phones. Only calls between her and I when she used Bluetooth and we were also NOT on the same cell tower would it fail. One of us couldn’t hear the other. The Verizon store even swapped my phone for a new one with the same result. They first tried the new phone with a different phone number and it worked but when they put my number on it, no go.
What finally fixed it was my wife deleted my number from her contacts and suddenly it was fine. Something must have gotten saved under the hood that messed things up. After it worked, she created a new contact and all was well.
I know this was on much older phones but your issue sounds just like ours.
After reading opnspaces suggestion, he’s recommended exactly what happened to work for us.