Taking off while the other occupants sleep

I'm an early riser and my family is not. In the past, our travel days would consist of me waking up between 6:00 and 6:30 and waiting for the family to arise at 8:30. Then we would have breakfast, p...