My daughter lived on the Mississippi Gulf Coast for a couple of years, ten minute walk to a beach on Mississippi Bay. I visited often, mostly in the cooler months.
Many folks go to this coast for the off-shore casinos. While there are outdoor activities in coastal swamps, winter is not the usual season for these attractions. Any place you visit has at least a little bit of history, but most of the historical sites in southern Mississippi are somewhat more inland, Civil War battle sites and preserved plantations.
Expect weather to be cool and wet at Gulfport during those months, lows in the low 40s, highs in the low 60s. Gulfport gets rain year-round, averaging about six inches a month. In January and February it tends to be more of a constant grayness and lighter rain, rather than the drenching and flash flooding storms of spring.