I agree with Francesca. Just go about your business. If he runs to you, afraid, give him a pat on the head and tell him its all good, and go on. They feed off of you.
In my opinion, for whatever it is or isn't worth.... I have always believed in exposing them to as much as possible when they are young. Getting them used to the sounds of the hair dryer and clippers, toe nail clipping -- as soon as their eyes are open. Normal household sounds. I've never let puppies leave before 14 weeks. I think that extra time with mom and littermates is important to their development.
Also, taking him back out with the calm, no fears dog was an excellent idea.
I've seen dogs who were raised in a barn when young - they are a mess. They are afraid of dishwashers, tv's, doorbells, vacuum cleaners..... it's better they get used to noise and situations when they are young. Makes for well adjusted adults.