Statistical abuse 101: College football edition

As the visiting captain, Houston tight end Christian Trahan called the coin flip Saturday as the Cougars entered overtime at the Alamodome against UTSA.

“Tails!” Trahan yelled.

“It is tails,” referee Hank Johns said. “Offense, defense or…”

Trahan cut off Johns before he could finish. The Cougars had waited almost two years for their captain to be able to say what Trahan said next.

“Offense,” Trahan said.

“You want to be on offense,” Johns repeated, before turning to UTSA quarterback Frank Harris. “What side of the field do you want to play on?”

Wait, what?

Johns didn’t sound surprised while arranging the players in the directions their teams would face, but Trahan reported to Houston’s coaching staff that officials had asked him multiple times if he really meant to say offense.

Hardly anyone chooses to play offense first in a college overtime game. The conventional wisdom dictates that it’s better to play defense first so the offense knows exactly what it needs to do when it gets the ball.

But after a presentation from Houston general manager Ryan Dorchester, coach Dana Holgorsen and his staff decided during the 2020 season that they wanted to play offense first. All they needed was an opportunity to make that choice.

That chance finally came Saturday.

This doesn’t appear to make any sense on its face. The college football overtime rules are that the teams alternate possessions from the other team’s 25 until one team has scored more points after each team has had the same number of possessions. The “conventional wisdom” here is based on the obvious fact that, when it comes to deciding what to do on offense, it’s an advantage to have more information rather than less. It’s the same reason why it’s an advantage to bat second in the deciding inning of a tied baseball game (there are strategies that maximize the odds of scoring a run while reducing the odds of scoring more than one run, so it’s crucial to know exactly how many runs you need to score to win), and an advantage to bet later in poker hand rather than earlier (the betting decisions of players betting earlier in the hand convey useful information to players betting later).

So what led Houston to make this extremely counter-intuitive choice? Machines, Gloster! I’ve got a whole room full of ’em:

Dorchester thought back to Oct. 3, 2020. He spent that afternoon in his living room, drinking beer and watching college football. It’s highly unusual for an employee of a college football team to do that on an in-season Saturday, but Houston had yet to play a game because of COVID-19-related cancellations. The Cougars would finally open their season the following Thursday against Tulane.

Dorchester was watching Baylor and West Virginia go to overtime in Morgantown. Baylor won the toss and chose to play defense first. West Virginia won the game in double overtime. A week earlier, Dorchester had watched Texas Tech force Texas to play offense first in overtime, and the Longhorns had won. That got Dorchester thinking: Why do teams always want to play defense first?

“Everybody does it. We do it blindly,” Dorchester said. “I wondered if it actually gave you a better chance to win.”

So Dorchester scoured the box scores of every overtime game played since the NCAA added overtime in 1996 to see if playing defense first really gave a team an advantage. The box scores get pretty thin in Web 1.0 territory, so when Dorchester couldn’t find detailed play-by-play, he reached out to sports information directors to fill in the gaps.

And since 1996, it has paid off. Dorchester, who has since updated his numbers to go through the 2021 season (781 overtime games), found that the team that played defense first in overtime won 52 percent of the time.

But that has changed over time. In the past seven years, the team that plays offense first has won 54 percent of the time. Dorchester took that information Holgorsen. “I think everybody’s got this wrong,” Dorchester told Holgorsen and Houston’s coaches during a staff meeting. Then Dorchester presented the numbers. The assistants had a few questions.

Holgorsen listened and issued a ruling.

“Makes sense to me,” he said. “This is what we’re going to do, boys.”

In the past two seasons, the numbers have lurched further toward the team that plays offense first in overtime. That team won 57 percent of the time. Dorchester has two theories as to why.

“First, teams are just more aggressive now than they were 10, 15, 20 years ago,” Dorchester said. “Coaches are more aggressive.”

This is true. Analytically driven coaches are more apt to go for it on fourth down. On earlier overtime downs, they’re more likely to play for a touchdown than a field goal attempt. Most coaches used to be run-first, but now many are pass-first.

The second?

“Pressure,” Dorchester said. “If you go out there and score and put pressure on that other side that they now have to answer*, I think that just does stuff to people.”

Whoa Nellie as Keith Jackson used to say.

So you do an analysis of a large data set that reveals exactly what you would expect it to reveal, given the conventional wisdom that all things being equal it’s better to have more information rather than less. (ETA: I am surprised that the apparent advantage of having the ball first isn’t larger — I would have expected it to be more on the order of 10% rather than 4%. That would suggest that there are other factors in play). But this doesn’t produce a conclusion that allows you, the sophisticated analyst, to give your coach counter-intuitive data-driven advice. So you cherry pick a vastly smaller sample of data that do appear to support the counter-intuitive advice. Then you construct some seat of the pants psychological theory about “pressure” that could of course be just as readily flipped around to reach the opposite conclusion, i.e., the team that has the ball first has more pressure on it because it doesn’t know what it needs to do.

Also too, note that the psychological justification theory simply assumes that the team that took the ball first has already scored a touchdown. In short “psychology” is the black box explanation that people like to cite when they don’t actually have a real theory.

The rain on your wedding day angle to this story is that, in last Saturday’s game, Houston kicked a field goal on fourth down and three yards to go after their decision to take the ball first in overtime, meaning they were conservative rather than aggressive, which is exactly the opposite behavior that taking the ball first is supposed to produce!

But they won, which I suppose proves you can prove anything with statistics.