I have been driving 2wd and 4wd dually's daily on an off for about 15 years. The only time I haven't been driving a dually is when I have one or the other of my excursions.
Once you get use to setting the truck with the mirrors they are more comfortable driving than any other truck. Sure I don't park next to handicap parking spot up front all the time, but it can be done. They will go through banks and fast foods drive thru's just fine. Just look ahead.
My wife drives my duallys all over too. Downtown is no big deal. If semi's and delivery trucks can go through so can you.