Yellow flame indicatetes one of two conditions. One is extremely rare (Sodium in burner area) the other is a bad air/fuel ratio.. This si adjustable but it's been a long time since I did that adjustment. And I do not feel like goign out to look at it.
Typically there are one of 3 adjustement
I think on the water heater.. If you look at the burner. just past the orifice are some windows cut in to the burner tube. There is a slider that "opens" or "Closes" the windows. it may (or not) be locked in place by a screw (or just friction) this is the adjustment. Slide it to .. I think Close the windows a bit (Do this whith burner lit) and you should see the flame blue up.
If you are not comfortable doing this.. Get professional help.