Do the two things that need to be powered really need 120V or are they using "wall warts" to provide DC from 120V? Most routers and modems use 5-12VDC. If that's the case, I would consider a "DC UPS" Something like
this or
this Much more efficient than converting 12V to 120V to convert it back down to 5 or 12V. There are lots of similar battery banks available, just make sure they can be used while they are charging as some of them are meant just to be used as a portable phone charger and not while they are being charged.