Plenty to do in Oak Bluffs. Ride the oldest merry-go-round in the country, walk the Tabernacle, have ice cream at Mad Martha's. However, if you do the steamship later ferries, you can make Edgartown which is a little more "Vineyardy".
The issue with the steamship ferries is reliability. You may get over and not get back. If you park in Woods Hole and have to fly back, it's a $60 cab ride from the airport in Hyannis to the ferry dock. I'm assuming that you don't have the flexibility to stay on the island.
The best (and most expensive) way is to just fly over and back.