We tried Florida, TX, the MS coast, and AZ. We ended up spending our winters in AZ in a small town (Bouse) about 25 miles from Quartzsite.
Regarding Valley Fever it can be serious if you have a weakened immune system, but for most people it is a non issue. The doctor we used while there said that if people go there for a few years most will catch it but not even notice or just think they have flu.
Once you have had it it rare to ever have symptoms again.
In ten years of spending part or all of the winter there we knew one guy that had a real problem with it. His immune system was compromised because of diabetes. It put him in the Parker hospital for 11 days. He was successfully treated with a drug that kills it, recovered in a few weeks and never had a problem after that and went back to Bouse for 7 more years.