How to listen to Audiobooks without a smart phone. UPDATE

I have a Sansa clip - every so often then go on sale at Best Buy for less than $30. They do everything in your list of requirements. It has a bookmark function for audiobooks and as long as you don't change the menu selection to anything else will start where you left off without having to go to the bookmark. Very small and takes a micro SD card in addition to its own large internal memory.

Another possibility but more expensive is an Ipod which also does all you are looking for.