With a slip fit at the axle and bearing are, and near zero preload as close as humanly possible, there is every likelyhood that the bearing will move a little. The weight of the trailer bearing down on that spot and the free rotation of the rollers within the race should keep them from moving but it's not a guarantee. There is nothing to locate them on the axle and keep them there other than proper tension and some prayers. If you used Loctite or similar you'd probably have a hard time getting them off next time. I have the same marks on my bearings. It's normal with the tolerances involved in today's trailers.
Sorry for the link. Puter won't let me access the tool bar. Maybe someone can make it a clicky for us.
http://www.nskamericas.com/cps/rde/xchg/na_en/hs.xsl/fretting.html