wheel bearing grease seal ?

So when I finished new bearings, and grease, and tapped on the grease seal , one side went in just farther than the other., just on one wheel. Should I knock this seal out and replace it perfectly? ,...