Should you take your passport YES, you never know when you might need it.
Is it worth crossing over into Algodones? For us only for cheaper dental, meds & optical, wouldn't cross just for a look see personally. Feels far more comfortable to be in than Tijuana we went to for lunch once, and that we'd never ever go back to for all the tea in china (threatened to throw hubby in the can about 15 years ago and scared the c*@p out of us all, as did stripping cars for drugs, ripping off panels, kids begging and packing bags in supermarkets = heartbreaking to see). If you like to go over to dine, drink or flea market type stuff, then go at 'er. For us no border town crossings into Mexico just for a look see except Algodones purely for meds and dental or aiming to get as far away from the border towns as possible for a resort area and with a group for our comfort levels.
Once bitten ...................
We aren't against Mexico totally even after all we read and personally experienced. In fact we look forward one day to caravanning with some others down the Baja, that we postponed from doing this past winter due to the looting after the hurricane last year. Didn't want to tempt fate right now.
It Takes No More Effort To Aim High Than To Aim Low - Reach For The Stars