You did not say how old the trailer is but an awful lot of trailers have been produced with manually adjusted brakes and then lots more with reverse adjusting brakes. The newer forward adjusting brakes tend to be better but a yearly check when you repack the bearings and put a new seal in can give you an insight into how well adjusted your brakes are. You seem to have one that was adjusted too loose compared to the others and that is why it still has material left.
When was the last time you checked them?