Use a voltage meter inside the transfer switch. You will see income and outgoing power. Check it while on shore power and you will see power top and bottom of the switches. I would leave the shore power plugged in and turn on the genset. You should have a noticeable switching of power and the shore power will be disconnected. If your generator input has power at the transfer switch but no transfer took place the transfer switch is bad. If you still do not see any power in that box other than shore power your problem is elsewhere.
Please be careful when working with live 120V connections.