jabres,
We have lived in the Grand Haven area for over 75 years and all the towns you had mentioned are lot alike with being beachtowns and so much to do fo all ages. All are very kid friendly and safe for all ages with St Joseph being the lowest on totempole for safety because of an "iffy" town on the northside of the river.
For example: The Grand Haven area alone has 12 campgrounds with around 1700 campsites total and they range from state parks/campgrounds (roughly 200 feet from your RV's door across the suberb sand beach to and under the Lake Michigan water and it has a very nice new fine playground too) to very luxurius campgrounds. About 1/4 mile from the state Park CG by way of the boardwalk along the river or drive on the road above with lots of free parking that goes to downtown (.7 miles) thru park after park with flush restrooms along the way and has about 70 nice couchlike benches, is the huge fancy free kids "Imaginination Station" kids play area of roughly 20,000 sq ft size in Mulligan's Hollow. It's where the YMCA and the Snow Ski Bowl is also and has lots of picnic tables as it's in the sand dunes on 3 sides and the river on the 4th side. On the boardwalk is also the nice free kids public splashpad with water jets that is very popular. So many other things to do and you don't even need to take your wallet along either.
The other 3 towns you'd mentioned also have various CG's and lots of amenities also and are family friendly too.
We know Ludington and Poncho's Pond well as we own acreage and also own 5 campsites in a member only park since 1987 a few miles inland from Ludington south of US-10. RVing, camping, boating, fishing and beaches plus forests and hiking is a way of life in West Michigan and we have over 75 years now of enjoying it as that is where we live.
However, be advised that all the CG's along Lake Michigan are very full all summer long and you must have reservations.
You will have to match the CG and area desired for your wants and/or needs as no mention of your RV or your kids ages or other things you want to do also and just too hard to guess at.
A superb CC LB 4X4, GM HD Diesel, airbags, Rancho's, lots more
Lance Legend TC 11' 4", loaded including 3400 PP generator and my deluxe 2' X 7' rear porch
29 ft Carriage Carri-lite 5'er - a specially built gem
A like new '07 Sunline Solaris 26' TT