OP here.
I think I may have figured out what the problem is.
It's not the battery. I did not charge it before bringing it in and it was fully charged and had a CCA higher than the sticker.
I think it is my clamps. They are top post type, and I think they are corroded. Reason I think this is due to the fact that jumping the battery from clamping on the outside of the clamps gave me power.
I am going to switch them to the ones that have the washer like connector that the bolt keeps it down to the battery. If that doesn't work, I will be hunting for more ideas.