I have to help xfer my wife exactly as you do OP. Been doing it for close to 20 years now. In all that time, I haven't had one person complain about me being in the ladies room.
I prefer to look for family style rest rooms. Luckily, we have a modified motorhome so haven't had to encounter issues on the interstates and rest areas. Anyway, I find that about 75% of supposedly handicap bathrooms are nothing of the sort and can't accommodate a person in a wheelchair.
I thought about making an app at one point to locate family rest rooms for travelers for this situation.
