The main reason some don't advise it is slop. You can't get a perfect slop-free fit between any shank and the receiver, and a reducer sleeve adds another fit and another source of slop.
There is no law against using the reducer sleeve. You won't get pulled over and ticketed, arrested, admonished, or otherwise inconvenienced. It is simply that SOME sources advise against using the reducer sleeve because of the slop.
If you use the reducer sleeve and notice excessive clunking, that's your problem. If you use the reducer sleeve and one day can't get the pin out because it's bent, that's your problem.