Ball mounts Is it legal to have one when not towing?

I don't leave my hitch in for the sake of my shins, I also think it looks better without the hitch sticking out. I have started locking the hitch when it is installed since one got stolen last year while camping, fortunately I noticed the day before we were to leave.

There are a couple of guys at work who leave theirs in all of the time and they don't even pull anything. Must be for the other cars all though the last time I was rear ended was almost 30 years ago.