OP with another update.
Had a mechanic come to my place with his bore scope. He could not find any contact inside the turbo. After about an hour of trouble shooting found the AC compressor and the clutch bearing for the AC compressor were both at fault. The additional load made the truck feel like it was losing power. The bearing failing gave me the grating sound I heard after a few minutes of idling.
I also had a boost fooler chip in the truck. This mechanic seemed to think a number of other factors were also in play. The chip was on its way out, I may have picked up some bad fuel, and I also needed to clean my air filter. It was almost a perfect storm of things going wrong.
I had the AC compressor replaced yesterday and it seems to run fine now. I haven't had the cance to give it a real test, but I should this week, I have 2 fivers to move. One is mine and weighs just 14k lbs. the other is BIL and weighs close to 10k lbs.