If you want laid back take a look at
http://www.geigerkeymarina.com/ It's near mm10 so a short ride into Key West. Has a restaurant/bar onsite right on the water. It's not fancy but we loved it. You can book fishing trips at the site, or fish from your RV site.
I just look at their site and it has had a facelift since we were there. Still the same laid back, friendly place. Watch the video for a feel of the place.
jim