...I thought that I had a handle on oil supply/demand, commoditization of, and why prices for crude (and, refined products therefrom) went insanely out of touch with reality and the consumer market.
I was so far wrong, it was embarrassing.
I would suggest everyone concerned with oil and derivatives pick up a copy of: "Oil's Endless Bid; taming the unreliable price of oil to secure our economy", by Dan Dicker, 2011, John Wiley & Sons (publishers).
What you ask about (diesel, and fuel in general) is in the realm of what is called, "crack spreads" ("...the relationship between crude and refined products"). Crack spreads of 3:2:1 ratios were used to to represent more the reality of refining crude to usable fuels from the 1980 to the end of the '90s; but more recently, trading of the FUEL (or, in the trading lingo: cracks) became more important than the ratio of crude to cracked products, and recently, a 1:1 SIMPLIFIED crack spread started being used (which absolutely DOESN'T represent the reality of refining. When refining, a general number representing the ratio of cracked fuel product one can derive from "a unit of sweet crude oil" is: for every 3 units of sweet crude oil, roughly 2 units of gasoline (3:2) are derived; and for every 3 units of sweet crude oil, roughly only 1 unit of distillate diesel (thus 3:2:1) is generated. Thus, in this book, it is pointed out HOW MUCH MORE IMPORTANT gasoline is in the refining process, than the distillate diesel, thus the twice as important fact that gasoline (sales) is TWICE as significant to refiners than diesel (gasoline is twice as widely produced as diesel). Now you understand how insignificant diesel distillate is in the big scheme of things from the refiners.
OK...FUEL market trading: according to the author, fuel traders always maximize their positions, especially during times of strong demand (like summer; and extremely or exceptionally cold periods during winter, and during time of huge economic pushes globally). With this in mind, and understanding that say, going long in a time of heavy demand, the MOST IMPORTANT time to attempt to drive up "settlement prices" is at the end of the day (the closing: settlement prices at the end of the day are used to "universally value ALL OPEN POSITIONS, thus the end of day prices are the most significant price for traders (these are the benchmark priced printed in the newspapers at end of day). So, if you can drive the price up (or, down) at closing, a few traders can wield a tremendous psychological pressure on the fuel market (inside the rules, AND LEGAL). This is where an outsized influence by a few traders can create unreal fuel prices in relation to the "normal range of settlements during the day, before the 2 minutes at closing". In the end, customers mostly just want to capture "the FINAL settlement prices", by doing an MOC (common market on close). After customers were burned by MOCs, the exchange created another trade called the TAS (trade at settlement), but TAS gave a huge edge to the floor traders (close to the action) rather than leveling the playing field. Floor traders would create a situation where the TAS lots they sold (earlier committed to sell) in the day have to be matched with the TAS lots bought late in the trading day (inspiring advantageous settlement: you've helped jack up)...so the writer shows that it is impossible "to level the playing field among participants in ANY free market". So, the uncontrolled oil market and distillates trading rolls along...
Anyhow, I suggest reading this book. It is fairly complex (far more involved than I attempt to synthesize above, as a hack amateur in the energy commodities market), and requires great concentration; but if you really what to know just how the uncontrolled energy market is destroying our economy and delaying/killing recovery, do find a copy...
Silver-