The Analytics Suggest Tamping Down the Hype on Houston

Here’s the thing: coaching matters in college sports. It should not be assumed that teams with great coaches will revert to the mean.

For example, since 2003 Gary Patterson is 37-20 in games decided by 7 points or less.

If you had kept betting on him to revert to the mean, you would have lost.

1 Like

I went to the www.thepredictiontracker.com to see how some of these “bottoms up” per play/per drive analytical guys are doing (FEI, Connelly, Feng’s Power Rank, ESPN FPI, Massey-Peabody).

Surprise surprise:

  1. Bill Connelly and Massey-Peabody don’t submit their predictions.
  2. The ones who do are no better than the best “score” based models (Dokter Entropy, Sagarin Predictor, Stat Fox, Pigskin Index)

Note: bigger “winners” is better (obviously), but smaller “error” is better (i.e. closer to actual margin).

2012
Updated line = 76.3% winners, 237 error
Sagarin Predictor = 76.5% winners, 252 error
Pigskin Index = 75.7% winners, 249 error
Dokter = 74.9% winners, 244 error
FEI = 74.6% winners, 307 error
StatFox = 74.2% winners, 263 error

2013
Updated line = 77.0% winners, 236 error
Sagarin Predictor = 76.6% winners, 257 error
Dokter = 76.5% winners, 243 error
Pigskin Index = 76.3% winners, 240 error
Stat Fox = 76.3% winners, 253 error
Feng Power Rank = 75.2% winners, 273 error
FEI = 75.1% winners, 288 error

2014
Updated Line = 73.7% winners, 250 error
Pigskin Index = 73.3% winners, 268 error
Dokter = 73.0% winners, 257 error
StatFox = 72.9% winners, 285 error
Sagarin Points = 72.9% winners, 272 error
Feng’s Power Rank = 70.5% winners, 274 error
FEI = 68.8% winners, 302 error

2015
Updated Line - 78.0% winners, 250 error
Dokter = 78.1% winners, 256 error
Sagarin Points = 77.8% winners, 266 error
ESPN FPI = 77.7% winners, 263 error
Stat Fox = 76.2% winners, 279 error
Pigskin Index = 75.8% winners, 261 error
Feng’s Power Rank = 76.0% winners, 276 error
FEI = 75.3% winners, 295 error

1 Like

YIPES … reading all those yearly results can be mind numbing …

As I stated earlier … making a point … THE HARD WAY …

And I only use the Vegas odds/stats/writeups to attempt to win delmarred’s weekly football pool … some use the eight ball … and others their lucky standing liberty quarter

:relaxed:

I’m really glad you guys are replying to this with mostly-productive discussion. CFB is really a uniquely-interesting sport when it comes to data analysis.

First, a little technical nitpicking – all CFB data (or at least the vast majority of it, depending on what you consider CFB data) is observational. CFB doesn’t do controlled experiments. You probably meant subjective data, and I’m going to respond assuming that’s what you meant.

I’m generally in agreement with your point that this particular data set (recruiting rankings) is bad, but I feel the need to make clear that subjective data does have a lot of perfectly valid uses in areas like medicine, economics, and marketing. If we could theoretically get some sort of recruiting-blind recruit rankings, we could probably use those, but that’s not a data set that currently does or will likely ever exist.

You’re doing a huge disservice to UT’s schedule here. Taking OU into 2OT and leading Bama late in the 4th both count for a lot in my book – maybe they shouldn’t, but I have yet to be convinced of that. Likewise, Northwestern was better than “above average” – going into the bowl game (where Tennessee obliterated them), their resume was comparable to that of UH. And putting UGA, Florida, BGSU, and Pitt all on the same level kind of suggests that your analysis is willfully ignorant.

You’re responding to a different argument than the one that I’m making here. I’m not making a regression to the mean argument – my argument is that the Cougars made a couple of high-leverage plays, specifically in the Memphis and Louisville games, that had a disproportionately large effect on the perception of their season and that weren’t necessarily probable. Essentially, UH can block the same number of Field Goals and Intercept all the same passes, but if those two blocks are made against, say, Tulane instead, the season’s outcome is dramatically different. Essentially, while blocking a Field Goal isn’t luck, blocking those two specific Field Goals probably is.

Let’s say, for the sake of simplicity, that the rate at which UH blocked Field Goals last year (2/17, unless you have better statistics that you’d like me to use – I can’t find very good stats on blocks) is exactly equal to the probability that UH would block any given Field Goal last year, and that the probability of a kicker making a Field Goal is equivalent to that kicker’s FG% from the range at which the kick was attempted minus the block that UH made.

Under this (admittedly oversimplified) model the probability that FG outcomes in those two games result in as good of a result for UH is

(1 - (15/17)^2 + (15/17) * (3/11) + (15/17) * (1/2)) * (1 - (15/17)^3) = about 28.28%

Admittedly, that’s a higher number than I intuitively thought, but it’s still not great. It’s certainly not a thing you can expect to happen repeatedly. So unless you have a particularly persuasive reason for me to believe that UH should be expected to execute better on game-tying/winning kicks in particular, I’m going to chalk that up to luck.

OK, so I was wrong about relying on Freshmen, and conflated JuCos with Transfers. I still have concerns about the amount of talent UH lost, but they’re not relying on Freshmen and JuCos.

Analytics people that have lifted up trophies:

  • RC Buford (Spurs)
  • Donnie Nelson (Mavericks)
  • Bob Myers (Warriors)
  • Basically everyone that’s won the World Series in the last decade
  • Stan Bowman (Blackhawks)
  • Dean Lombardi (LA Kings)
  • Jim Rutherford (Penguins)

Yeah, you’re right. Analytics is a load of garbage.

For the purposes of this discussion, I’d classify Sagarin, StatFox, et al. as “analytics” – a term here meaning “sports analysis that makes traditionalist meatheads mad because they skipped math class and now the nerds are taking their jobs.” The article in question here was talking about Analytics as opposed to “Expertise” – that is, instead of comparing to Sagarin and StatFox, the better comparison would be to the likes of Phil Steele and Lindy’s.

Furthermore, the models that you advocate here also have the Coogs ranked similarly low. Dokter has them at 37, and Sagarin is similarly unlikely to be kind, given that they were ranked 21st to end last year.

Lets get the facts straight here…NO Louisville FG would have won the game…It would have tied the game…and as Houston had outplayed Louisville this game there would be every reason to think that even if the FG had been good, Houston wouldve still won that game…But no need…Our guys made a big play on defense and Houston wins…Thats not lucky…
Every great team usually has one game where they get a break or 2 to help secure that win…For us it was the Memphis game…Thats 1 win out of 13…We won last year because we had good talent and excellent coaching.

Analytics or no analytics… although the DL is stout with returning seniors, replacing four year starters, leaders and chemistry takes more than a single summer of practice
As such it is plausible that UH loses key games …falling to Oklahoma and Cincinnati (away game) and this year’s team has a huge question mark.

On paper UH has won games they should not have and they have lost games they should not have … 'nuff said!
Next question?

Coog2088:
UT beat NW (above average), UGA (average), and BGU (average) but lost to 2 average teams in UF and Arky, while Navy beat USF (average), Memphis (average), and Pittsburgh (average), but only lost to 2 of the best 2015 teams in CFB in UH and ND (although they were blown out in each).

T-Moar:
You’re doing a huge disservice to UT’s schedule here. Taking OU into 2OT and leading Bama late in the 4th both count for a lot in my book – maybe they shouldn’t, but I have yet to be convinced of that. Likewise, Northwestern was better than “above average” – going into the bowl game (where Tennessee obliterated them), their resume was comparable to that of UH. And putting UGA, Florida, BGSU, and Pitt all on the same level kind of suggests that your analysis is willfully ignorant.

Coog2088:
Who did UGA beat this year? The SEC east was very bad this year. Looking back at the schedule UF had a better year than I thought. UGA wins = ULM 2-11, Vandy 4-8, USCe 3-9, Southern FCS, Mizz 5-7, UK 5-7, AUB 7-6, Georgia Southern 9-4 in OT (best win), GTech 3-9, PSU 7-6…… Explain to me why I cannot put UGA, BGU, and Pitt in the same category of average cfb teams for 2015. Average means beating the bad teams and losing to the good teams right?


Coog2088:
Steven Taylor blocking a field goal is not luck. Adrian McDonald catching an interception versus dropping one is not luck, it is a skill that he possesses. The players executed, plain and simple. Saying they SHOULD have regressed to the mean is irrelevant to the 2015 and 2016 Cougars, and not a reason to downgrade them.

T-Moar:
You’re responding to a different argument than the one that I’m making here. I’m not making a regression to the mean argument – my argument is that the Cougars made a couple of high-leverage plays, specifically in the Memphis and Louisville games, that had a disproportionately large effect on the perception of their season and that weren’t necessarily probable.

Coog2088:
Who cares if it was probable or not? It happened. UH made the right plays at the right time, which is a key component of championship caliber teams. One play can make or break a season, so of course a field goal at the end of a close game is disproportionate.


T-Moar:
Essentially, UH can block the same number of Field Goals and Intercept all the same passes, but if those two blocks are made against, say, Tulane instead, the season’s outcome is dramatically different. Essentially, while blocking a Field Goal isn’t luck, blocking those two specific Field Goals probably is. Let’s say, for the sake of simplicity, that the rate at which UH blocked Field Goals last year (2/17, unless you have better statistics that you’d like me to use – I can’t find very good stats on blocks) is exactly equal to the probability that UH would block any given Field Goal last year, and that the probability of a kicker making a Field Goal is equivalent to that kicker’s FG% from the range at which the kick was attempted minus the block that UH made.

Under this (admittedly oversimplified) model the probability that FG outcomes in those two games result in as good of a result for UH is
(1 - (15/17)^2 + (15/17) * (3/11) + (15/17) * (1/2)) * (1 - (15/17)^3) = about 28.28%
Admittedly, that’s a higher number than I intuitively thought, but it’s still not great. It’s certainly not a thing you can expect to happen repeatedly. So unless you have a particularly persuasive reason for me to believe that UH should be expected to execute better on game-tying/winning kicks in particular, I’m going to chalk that up to luck.

Coog2088:
Why are you combining the season as if it were one game? Each game is independent of the other because the opponent is different. What UH did against UL does not carry over to the next opponent. For 2015, we made the plays we had to make, when we had to make them. I don’t care if you run the simulation 4 times and we only block the kick once. Each play in a season is not weighed the same. Example, when Kyrie Irving hit the 3pt’er to win the Finals was it luck? He only hits 40% of those shots. NO it’s not luck. The specific play is not lumped together with the other plays in that season, or even that game.


Regarding the 2015 UH Football team:
We stopped the run, ran the ball, and made smart football plays. We did not turn the ball over (which is NOT luck. An opposing team’s DB dropping a pick is not luck. He was not good enough to catch it) and we FORCED turnovers (which is also not luck because the same players did over multiple years=skill).

When predicting the 2016 season:

  1. Our returning starters played very well last year.
  2. Our coaches have proven they know how to put the players in the best position to achieve and win games (albeit a small sample size). The only reason I could see someone doubting the upcoming season is because we have so few returning starters compared to other teams. But IMO a returning Heisman candidate QB cancels a lot of other positional players’ mistakes.
    That’s all that goes into the model.

Something I would like to point out here as well. For analytics purposes a field goal at the end of the game doesn’t work. The reason being that a disproportional amount of last minute field goals would be ones you wouldn’t normally go for, but it is your last shot so you do it anyway. This means MORE of them will be missed and MORE of them will be blocked.

Using that logic, EVERYTHING you do in life is luck. the fact that we scored on Cincinnati is bad Luck cause those scores could have come against UConn instead… or the fact that all of our guys didn’t get hurt in the Louisville game is luck.

The fact that there are a million other possible outcomes for what actually happened, does not mean that what happened is luck…that’s a seriously thin and convenient argument.

1 Like