You need a notorized consent letter from each parent, or that both parents have signed, and passport. The consent letter says that the parent is allowing you (by full name), your spouse (if applicable, also by full name) to take the child (by full name) out of the country, the dates, and that the parent also gives you permission to get medical care for the child.
The one that we use (my wife and I do separate letters for each kid), which has worked fine for several years now when my parents take my kids to Canada:
{Name of father}
FATHER OF {Name of child}
{Address of father}
This is a letter of consent permitting {Name of child} to travel to Canada {start date} and return to the USA {end date} (dates may vary slightly due to unknown circumstances) with his grandmother, {Name of grandmother}, and his grandfather, {Name of grandfather}.
If needed, grandparents, {Name of grandmother} or {Name of grandfather}, also have my permission to seek medical treatment for their grandson, {Name of child}, during the time of {start date} thru {end date} (dates may vary slightly due to unknown circumstances).
Signed:
__________________________________________
{Name of father}, father of {Name of child}
__________________________________________
Date (month, day, and year)
__________________________________________
Home phone number
__________________________________________
Work phone number
__________________________________________
Cell phone
Sworn to before me and subscribed in my presence this _______ day of ________, 2014.