We stayed there in 2013 and our kids were 10, 8, and 4; so a little older than yours but 3 and 4 are pretty close. You can easily spend an entire day at the beach. Bring your lunch, some sand toys and kites. The kids will love it. Just playing in the sand is a great time.
Make sure you visit the beach at least once during a sunset.
If you like to ride bikes and have a bike trailer for the little ones, bring it and go to the state park. There is a really nice paved ride that loops from one side of the river to the other and crosses over a dam in the process. You can fish at the dam or on the lake side. My kids enjoyed all of these.
In town, visit House of Flavors for ice cream. It can have a line out the door after dinner, so time accordingly.
Right down the street from the ice cream shop is a marina where you might enjoy walking down the piers to look at boats.
There is also a wonderful park. Seriously, your kids will love this place. It is right next to where the SS Badger comes into port. It departs and arrives once a day and if you time your stay right, the kids can watch the freighter come in. That's always fun b/c the park is literally on the water.
Ludington is 1.5 hours from sleeping bear dunes if you decide to go. As a family, we liked it, but we prefer other places. It's probably not the best place for young children, but the adults might like the view.
The part my kids liked best about Sleeping Bear, was visiting the store Cherry Republic before we left. Everything cherry, and so good with free samples. It's worth going back just for the store.
Our favorite place in the area is Silver Lake Sand Dunes. We have a dune buggy and have been going there for 18 years so we know it well. However, this park is not a place for walking with young children. To experience the park, you really have 3 choices. 1-Take a ride with MacDune Rides. 2-Rent a dune buggy, quad, or side by side. 3-Drive in on your own 4X4 truck. If those don't appeal to you, you'll never really experience the best of it and shouldn't bother visiting the area.
Have a great time.