It was like that in 2005 when we visited. We were early enough (June 6) that with the snow they had that year that the valley was just about the only thing open which made the crowds even worse as they had no place to go. We had reservations in Wawoona CG for four nights, but after being in the valley for a couple of hours, we had enough and left after the second night.
Part of the problem was we had also been spoiled by spending the five previous nights in Azalea CG over in Kings Canyon which was pretty deserted. It was nothing for us to go down into the canyon and only see a couple of cars all day.