Why did it take so long for “analytics” to have an impact on the multi-billion dollar world of professional sports?
By 
That’s the basic question that Andrew Gelman is asking in this post. He suggests one answer, in the specific context of American football:
I guess part of the problem is, to use some psychology and statistics jargon, a cognitive bias induced by ecological correlation. There always were some teams that tried unconventional plays, but they tended to be less successful teams that tried these tactics as a last resort. The Oklahomas, the Michigans, the Vikings and Steelers didn’t need this sort of thing. The only thing at all out of the ordinary I can remember being routinely played is Dallas’s two-minute offense with Roger Staubach in the shotgun, but that was a rare exception, as I recall it.
Consider a sequence over the decades:
1. Tactics are developed during the play-in-the-mud, Army-beats-Navy-3-to-0 era.
2. Conservative coaches stick with these tactics for decades.
3. Spectators are so used to things being done that way that they don’t even question it.
4. Analytics revolution.
5. Even now, coaches shade toward the conservative choices, even when stakes are high.
We’re now in step 5. In his above-linked post, Campos expresses frustration about it. And I get his frustration, as this is similar to my frustrations about misconceptions in science, or clueless political reporting, or whatever. But what really intrigues me is step 3, the subject of this post, which is how we were so deep inside this particular framework of assumptions that we couldn’t even see out. Or, it’s not that we couldn’t see out, but that we didn’t even know we were inside all this time.
I wonder what Gerd Gigerenzer, Daniel Kahneman, Josh “hot hand” Miller, and other experts on cognitive illusions think about this one.
The specific suggestion here is that tactics that might make sense in much lower scoring eras cease to make sense when scoring becomes higher, but neither coaches nor fans adjust to the new reality, or adjust very slowly.
This explanation doesn’t really work for the NFL, since scoring in that league has been remarkably stable for the entire post-WWII era. When we look at NFL scoring averages, it’s obvious that the game’s rules makers are constantly tweaking the rules to maintain a balance between offense and defense that results in a scoring average of about 20-23 points per game per team, with significant changes being made whenever — such as in the late 1970s when pass blocking rules were liberalized — scoring begins to fall outside this very narrow range. (By way of contrast, scoring averages in major league baseball have varied far more over time).
The most obvious cognitive illusion/failure that gripped football coaches for many decades, and has only recently begun to loosen, was the tendency to punt far more often than even the crudest statistical analysis would consider optimal. Here’s a passage from my new book A Fan’s Life:
I remember the precise moment when my frustration on this topic reached a boiling point: October 30, 1999. Michigan was playing Indiana in Bloomington, and the Wolverines featured an offense made up almost entirely of future NFL players, including eventual superstars such as quarterback Tom Brady and offensive lineman Chris Hutchinson. Meanwhile, Indiana’s quarterback Antwaan Randle El – also a future NFL star – was that afternoon a kind of offensive wizard, whom Michigan’s defense strove to contain with little success. Tied late in the third quarter, Michigan was faced with fourth down and one yard to go on the Indiana 41-yard line. To my amazement (this is a rhetorical phrase — I was completely unsurprised), Michigan’s coach, the classically conservative Lloyd Carr, decided to punt. Predictably, the punt went into the end zone, which gave Indiana the ball at their own 20—a net change in field position of 21 yards. Even more predictably, on the first play after the change of possession Randle El proceeded to gain back 19 of those precious 21 yards by eluding a host of befuddled Michigan defenders. So Michigan had given up the ball, foregoing an excellent chance of maintaining possession by converting on fourth down, just so Indiana could end up having the ball at almost exactly the same spot, one play later.
In so-called real life, I wanted to scream in frustration. Luckily, the [Michigan football message] Board, open on my laptop in front of the TV, allowed me to scream for the 200th time to my fellow sufferers that a punt is a turnover.
Thus I considered it an almost personal vindication when, a few years later, University of California economist David Romer demonstrated with mathematical rigor that coaches did punt far too often, given the relevant probabilities. Romer’s paper – which is technically about the more academically respectable topic of the extent to which firms maximize profit-making opportunities – showed that the conventional wisdom about punting was radically wrong. His conclusions suggest that coaches punt far too often because they frame losing the ball on downs as a bad thing, while interpreting giving the ball away via a punt as, if not exactly a good thing, then not nearly as bad an event. And of course it isn’t nearly as bad – if you ignore the opportunity cost of foregoing the chance of keeping the ball. But that opportunity cost is, Romer demonstrated, massive: so much so that coaches should go for a first down in situations where, according to the conventional wisdom, doing so would be considered the height of recklessness. As big-money sports become ever-more profitable and competitive, the cost of ignoring the insights of people like [Bill] James and Romer becomes ever-greater. Indeed, shortly after Romer’s paper appeared, it found an apparently unlikely reader: New England Patriots’ coach Bill Belichick. Belichick adopted much of Romer’s analysis to his own in-game decision making, with predictably successful results. (In 2004, I authored, along with my friend and fellow Michigan fanatic Jon Chait, a piece for the New York Times, in which we discussed Belichick’s use of Romer’s work.)
Belichick was an early adopter of going for it on fourth down rather than punting in a wide variety of situations where the conventional wisdom required a punt. In the ensuing 20 or so years tremendous progress has been made in this regard, although punting still remains over-used, so tenacious is the psychological hold of the factors that made it so unjustifiably popular.
The larger question here is, as Gelman notes, why it takes so long for commitments to various suboptimal tactics and strategies to deteriorate, in the face of compelling data demonstrating that sub-optimality? (An example from the world of baseball was the many decade-long failure to recognize the value of walks, and the related failure to grasp that the hitter is a far more important variable in whether a walk is issued than the pitcher).
Part of the answer, I think, has to do with the relationship between anti-intellectualism in general and the culture of big money sports in particular. “Analytics” were for a long time considered to be some sort of nerdy black magic, that represented a kind of violation of the Code of the Game, which could only be understood by Those Who Had Played It. This attitude still very much exists, though it has been beaten back somewhat by the Moneyball generation, as the competitive and therefore economic costs of willful ignorance and stupidity become higher. (“What do the analytics say?” broadcasters will now ask on fourth down situations, as if our computerized version of the Oracle at Delphi was about to be consulted).
The application of all this to realms outside of sports is, as Gelman also suggests, quite striking, although in a culture in which a revanchist fascism is celebrating ignorance and stupidity as positive virtues, practitioners of “analytics” in every field find constantly that, as Camus noted, their rock awaits them.