Guess I'd better not say in writing what I/we would do about it. But, If he continues and you are 100% legal with your fires, the law is on your side. I don't know all the details or particulars of your situation as that could have some bearings. For a start, file a written complaint and press charges and follow it up. Use the law and enforcement to shut him up!
Kind of strange that he can't open his south windows at all when you have a fire and is like saying you must have a big fan always blowing the smoke directly at his house as if the wind direction never changes. One BIG LYING MOUTH blowhard jerk for a neighbor!!! WE know the constantly varying wind directions here in the Midwest as we also live in the Midwest and have for over 74 years.
A superb CC LB 4X4, GM HD Diesel, airbags, Rancho's, lots more
Lance Legend TC 11' 4", loaded including 3400 PP generator and my deluxe 2' X 7' rear porch
29 ft Carriage Carri-lite 5'er - a specially built gem
A like new '07 Sunline Solaris 26' TT