Sometimes the battery backup alarm clocks don't last all that long on battery backup or keep super accurate time on battery power. Many, especially older ones, rely on the 60 Hz power line frequency for their basic timekeeping. The battery power just runs it at more or less the right rate until the (presumably brief and rare) power outage is over.
I think coolmom's suggestion is about the best: get a clock that's purely battery powered. The only drawback is that most don't glow all the time during the night like the big red numbers on many plug-in alarm clocks, so it's harder to know exactly what time you've woken up to go potty.
I use a
nifty little battery alarm clock in my RV from Ikea. It's about two inches square, and tells the time/date, the current temperature, the alarm time, and has a count down timer, depending which side it's set on. The only difficulty is that you can't easily pack it in a drawer on its back as the wiggling around of the RV makes it change modes more or less randomly and make beeps and buzzes and such. I generally tuck it under the corner of the comforter next to the bed mattress while driving and that seems to work well enough. The timer works very nicely for games like Boggle, too.