Hard to help without knowing what brand and model of your hitch. I had that problem once with my SuperGlide wrap around jaws. Found out that the plate was a very tiny bit distorted. this did not allow the jaw to completely wrap around the pin. Thus a less secure hold on the pin. I had to remove the hitch plate and straighten the are that was deformed. Only about a .015" deformity. But those hitches are pretty close tolerances.
In you case it could be a matter of high pinning, a burr somewhere, a mis-alignment, a lack of lubrication, or something else.
