Anyone upgrade their interior and exterior speakers?

You're trying to force a single cone to accommodate a wide variety of frequencies. At least go for a 6x9 with a separate tweeter in the center; some will even have a separate tweeter and mid in the center leaving the big 6x9 cone only for bass.

I wouldn't say marine is a necessity--there are a lot of good name speakers with polypropylene (instead of paper) cones that would resist moisture (but not rain) for a lot better price. You pay a lot for marine and don't necessarily get a ton more weather resistance.