is it just once or all the way down a gravel road? Gravel is bumpy, but so are paved bridges and pot holes and other paved road issues, so strange it is only on gravel.
How fast are you driving down the gravel?
The receiver hitch will clunk, but should not be a bang.
Is the bang heard or felt? Sounds like trailer or is it resonating through your vehicle?
Few things would have enough power to make a big bang sound. The WD hitch could, though I have never heard of this. If it is really really worn out that could be it. The bars are under tension and enough tension to have power for a big noise, but once unleashed you would have a damaged, broken or dangling WD bar hanging.
The trailer axle and springs have potential to store enough energy to make a big sound. Check that they are tight. If a leaf spring, then just tighten the U-bolts with a deep socket wrench to make sure they are tight. If loose then you'd need to make sure they are on straight still.
If the wheel was loose it would be gone by now.
If your camper top were not latched it would be gone by now and probably not enough there to make a bang either.
Does it fit snug, or is there still up and down play when latched?
Something big rolling around in the camper? unlikely as it is probably tightly packed in there, but only you know.
I suspect you do not have a WD hitch, so you might be bouncing or what they call pourposing (somebody can chek that spelling..) this could cause things to move.
Broken trailer frame?
Need more info, too many unknowns for us