The spray kills them, but it also leaves a lot of stragglers outside the nest looking for someone to blame.
Best way to get rid of hornets, ground bees, etc. is to put some Ortho Sevin powder in a ketchup bottle, and shoot the powder in and around the entrance of the nest. The bees don't mind the Sevin, and track the stuff into the nest, and within a day or two they are all dead, queen included. Saw an exterminator do this--took him five minutes including fetching a ladder from his truck, and he handed my neighbor a bill for $100. Quick, easy, and little risk of getting stung by angry bees.