Since you're trying to stay away from plywood, have you though about using the white PVC trim boards that Lowes or Home depot sells. They cut and plane down with standard wood working tools
You can slice them up just like wood and shim out the aluminum. It would act as a bit of a heat barrier between the aluminum frame (probably not a great insulator) but it doesn't rot.
Another item I've found that works well for "shims" is the slats from the 2" fake wood window blinds. They are made from PVC and are about 1/8" thick. PVC glue is quite cheap, you glue 2 together if you need 1/4".