I've got 2 panniers for each side of my bike which you can easily put on or take off. Also, if you want to only use one that is easily done. If you use panniers on the rear of your bike you also have to have the rear tire rack. The LL Bean page shows some nice handlebar baskets.
Here are some ideas.
http://www.llbean.com/llb/shop/8212?mkwid=sUuIaVpJd_dc&pcrid=30958848177&pkw=bicycle%20panniers&pmt=b&qs=3084924_googlehttp://www.jollyoutdoor.com/pannier-bags-trunk-bags/?gclid=CMDZ6MffproCFYxAMgoddWYAaA