During one of our trips through GA we stayed at a state park that had horse camping sites and stables: Hard Labor Creek State Park
http://gastateparks.org/HardLaborCreekIt was about an hour's drive east of Atlanta. Pretty nice cg but can't evaluate their horse accommodations as we have no horses.
If you ask about horse camping in the Parks and Campgrounds thread category you might get a number suggestions for cgs.