Just check with your State's ag office for rules/restrictions/advice. Problem differs from state to state, and even from area to area within States.
In my case, if camping here on the Olympic Peninsula, I bring firewood from my own property- usually just enough for a day or two, though. There's lots more available free for the cutting in the woods.
But if I travel OFF the Peninsula, I don't bring any wood from home just in case of contamination with some bad boy not yet occurring elsewhere. And I never bring firewood home from ANY distant source- just leave whatever I've gathered behind.