With the belt off the pulley you may notice a gravelly feel when turning it. Briefly start engine (briefly!) without belt and confirm noise is coming from one of the accessories. Do a water squirt on the belt just to rule that out too.
If try as you might you can't really narrow it down, you could always replace the idler bearings as periodic maintenance-especially if you plan to keep truck a while. Proceed from there if that doesn't work-sometimes at the very beginning of failure the noises are hard to locate.
