It's likely quite normal. It does depend some on the model of the converter; the older ferroresonant converter designs are almost notorious for humming, due to the big honking transformer in them operating at 60 Hz. Many newer converters are based around switching power supplies and are much less likely to hum as they have much smaller transformers operating at a much higher frequency.
It may also be fan noise, particularly if the fan is a variable speed fan (which is common), it might be noisier at particular speeds.
In short, so long as you're getting expected operation from of the converter, I would not worry about the noise.