Not every baby sleeps through everything. If the folks were polite about approaching you, I would have turned it off for a few hours - just so I wasn't listening to a cranky baby all evening. If they were rude in their approach, I would voice displeasure with their tone but agree to turn it off for an hour.
Is an hour or two with the AC turned off while you are gone really worth getting upset over? For me, it wouldn't be. I'd rather have a rested and happy baby in the site next to me than run my generator non stop during the day.