Had a business meeting in Charleston and went a little out of my way to eat at this recommended by a local, place:
http://167raw.com/Didn't want to leave. Ever.
It's prolly about 15-40 minutes outside of James Island depending on how you are going and where you are staying. They don't take reservations so try for a late lunch or line up for dinner time
early to get a table. It'll be worth every minute. One of the bestest buttered CT style (that means no mayo) hot lobster rolls I've ever had with recognizable chunks of lobby. Fried oyster basket was delectable, not too pale, not too brown; just right with spurts of salinity breaking through the fried crust. The shrimp tacos were nothing to sneer at either. A form of seafood heaven that makes good sense when I found out the owners originally came from Nantucket.
Within JI proper, give a try to Roadside Seafood for my money. They have a light touch with fried food and supposedly serve low/no gluten fries which I didn't try since I wanted to concentrate on the "main event".
HTH and where ever you go may it bring you good eats.