If I was planning this trip, I'd do the US part one year and Canada another. Your'e simply not spending enough time in each city!
I would do 2 days in Philly. Historical area, and dont miss East State Pen. Use the PHLASH to get around the area.
NYC...I can take it or leave it, but...1 day for Ellis Island and Statue of Liberty, maybe another 2 days to see the rest you want to see.
Boston...1 day for Freedom Trail then I'd hit Plimith Plantation and head to Cape Cod. See some of the sea shore, light houses, Provincetown. That's about 3 days pushing it. Maybe even see Salem.
I would spend the rest of the time in the White Mountains and/or Acadia.
If you do go through Canada and back through Detroit, stop in the Henry Ford. You won't be disappointed.