Sure the caravan for the first time would be nice, they bring you to all the nice place to visit and would get me and the wife more knowledgeable for the next time. I sure would not take it more than one time.
I am not normaly too afraid to go to places and I have been to the Baja from Mexicali. I was kind of surprised to see the army asking me questions on the road to Tijuanna and I saw a few very poor places on the way that was questionable if you had to stop.
On the other hand, I walked all over in Ensenada at night, going from town to the campsite, also in Rosarito and further down the coast. I came back using the inland road and did not find any problem there.