My oldest daughter (now 28 yrs old) took her first trip in the RV at 6 weeks old. Glamis. The noise of the ATVs and buggies running all night long meant she didn't sleep a wink the first night...which also meant she sacked out completely for night 2!!!! The actual traveling part was easier for us since we have a motorhome, not a trailer; they could play games at the table, nap in the back, etc.
My brother's son, at 3, got so bit up by mosquitoes one trip that he was hallucinating.
Otherwise, can't really think of any "oh no" situations with any of my 3 little ones (now 28, 25, 23), and they all still go camping with us, along with their spouses/significant others, and my 3 year old grandson! Just got back last month from the desert, riding ATV's/MCs with 4 generations, including my dad (85 yrs old) and my new niece (4 mo. old). Yup, he (my dad) started it...and we're keeping it going!