A structurally stiff box (so it doesn't resonate) will help a great deal without any sound absorbing material. A double wall with mechanical isolation (so the box doesn't transmit vibration to the outside world) would be even better. You really don't care how loud it is inside the box with only the generator there to hear it, so sound absorbing materials are less necessary inside.
For many small engines like are used on (contractor) generators, a muffler upgrade can yield a big improvement. A small automotive muffler often works well and isn't too hard to find for fairly reasonable money. (My little lawn tractor, on the other hand, has a reasonably effective muffler from the factory...that costs about $200 to replace. Such are the joys of a Toro mower with an engine made by Onan.)
