Dw is from Port Townsend, And I have spend a lot of time there so I probably know it better then most.
Pt would be lot's of fun. But I would skip the ferry, and drive around through Tacoma over the narrows bridge. Ferry is fun but you would have to catch it in Seattle, and maybe wait for a boat. So is kind of a time killer.
I would suggest, as you mention a walk, Fort Warden or Fort Flagler. Both are state parks and have excellent beach walks. As a bonus both have Large shore battery bunkers from WW1. They are fun to explore. Kids tend to really enjoy them. A flash light is a good idea