Fun with statistics, Scott Frost edition

I saw this stat and tried to do a little figuring:

The way win probability works as I understand it is that it’s a simple extrapolation of how often teams in general who are in X situation in a game eventually win that game. In other words, there’s no attempt made to adjust for the relative strength of the teams in a particular game, so it’s not something a gambler would use to determine probabilities. For example say that teams that are ahead 7-0 in the middle of the first quarter go on to win 63% of the time or whatever. That does not of course mean that anybody would think Rutgers has a 63% chance of beating the Ohio State football team if it were up by that score at that juncture. But it’s apparently a good predictor in general, since it’s purely empirical.

OK. I asked myself, by just looking at Nebraska’s highest win probability in those seven games, what were the odds that the team would win all seven? The answer I got was 45.6%. So that would make the odds of losing all seven . . . let’s see six minus three carry the one . . . um, about 25 million to one against.

Now there’s got to be some sort of statistical fallacy at work here — there’s no way the statement “Scott Frost’s results at Nebraska in games in which the team had a high in-game probability of winning would happen only once in 25 million times randomly” could be true now, could it? — but what is that fallacy exactly?

Here’s what I’ve come up with after consultation with redeye whiskey for the pain:

The 25 million to one stat is a product of extreme cherry picking, or what I guess is known more formally as a compositional fallacy. While it’s true that Frost’s record at Nebraska is marked by an unusual number of losses in both close games, and in games where the team had a very high probability of winning at some point during the game, choosing those seven games out the 45 he’s coached at Nebraska and considering only that subset of probabilities when asking how often Nebraska wins games relative to its highest in-game odds of winning those games creates a kind of illusory statistic.

The example that sprang to mind to demonstrate this is that Nick Saban has lost 25 games at Alabama, and the highest win probability for Alabama in the course of each those games was never less than 50% — and of course much higher at some point in many of them — so you could get some truly astronomical number if you calculated what the odds were of Alabama going 0-25 in those games, given the team’s highest win probability in each. Nevertheless Saban seems to be a pretty good coach, based on the other 178 games that he’s coached there.

So you may ask yourself: how did I get here? I don’t know, but I thought it was sort of an interesting demonstration of some general principle about statistical extrapolation, though I’m not confident I’ve described exactly what that principle is.

None of which is to deny that Scott Frost should be fired, even though Nebraska would owe him a tidy $7.5 million in buyout money if it were to do that– a sum that is half of what it was after his contract was “re-negotiated” in the midst of the team’s disastrous 2021 season. Speaking of crazy numbers, check out Nick Saban’s new contract extension, which pays him a minimum of $93.6 million over the next eight years, not counting performance incentives, because if there’s one thing Nick Saban really needs it’s performance incentives. Well that and free oil changes for his employer-supplied car (also in the contract).