If you peel the citrus fruit and leave the peelings, they'll let you take the fruit.
On one of our day trips into Alaska, the agent let me keep the whole fruit with skin, but I had to promise to NOT leave the peels/rinds in Alaska if we ate the fruit.
On a crossing from Canada into the Lower 48, the agents took my California grown oranges indicating they may have become contaminated while in Canada.
http://www.cbp.gov/importation-advisory-citrus-fruit-may-not-be-brought-into-USA "We just want to get the word out that these products are prohibited from being imported into the United States because of the unseen diseases and pests they may carry."