Saban speaks on Analytics

[RollTide_HTTR]

People, coaches, and organizations treat analytics differently, so I don't think we can make a general assessment of how the majority treat them. There are some people who let analytics basically "coach the team", especially in baseball. Then others, as you say, use them as a tool to get an edge. I've encountered both.

Yea that's definitely fair. But I think the larger analytics community is mostly trying to sell it as a slight edge

 

[stlimprov]

...says the guy who has more analysts on staff than most places have students on campus.

 

[KrAzY3]

Analytics looks at the odds of winning if you punt it, kick a field goal, go for it on fourth and make it, and go for it on fourth and make it. And then you make a decision based on that. The good models are also more detailed than that.

I think a lot of people assume there is no risk to kicking a field goal or punting but that's not true. I also think people misunderstand what analytics is doing

I'm not against analytics, but I am against doing things only with as 4Q alluded to, the law of big numbers in mind. If you have to win every single game, 90% success rate or what have you might not cut it. Also for the record I'm a big stats guy and always have been.

Anyway, to give an example, let's say the data says if you go for it you have a 91% chance of winning and if you punt you have a 90% chance of winning. Well, analytics says go for it. Ok, but let's say you don't make it and suddenly your odds drop to 80% chance of winning. Sure if you did it 1000 times you might come out ahead (or the model might not have the right data for your scenario), but you aren't in those other 999 scenarios, you're in the one where you just doubled your chance to lose.

So the counter to over reliance on analytics is always looking at what the worst case scenario is. What's the worst thing that can happen here? For instance Alabama has the ball on the opponent's two yard line, analytics might be (just give an example, not saying that's the case) fine with throwing a pass, but Alabama did throw a pass and it was intercepted and Alabama lost the game because of it and lost a chance at a championship because of it.

Also, no matter how good the model is, it can't possibly know everything. For instance would the model know Mac Jones was playing that game after Tua got hurt? Would it factor that into the analytics of whether or not it was wise to throw from the two yard line?

I've seen a few coaches turn close games into blowouts against Alabama because they kept going for it on fourth down and failing. I don't doubt analytics said go for it, but did the model really factor in it was the Alabama defense they were going against? In a generic game with two generic teams facing each other, it might be better to go for it on fourth, but against Alabama it might not be.

How could you even use analytics as a playcaller when you have Milroe as your QB? You don't know if you're going to get good Milroe or bad Milroe, I have no idea how well he'll play next game and there's no model that can predict that. So if I'm looking at an analytics model, and trying to figure out what choice to make, how do I factor in the unpredictability of MIlroe?

No model will ever be able to fully figure all the variables out, there's just no way to get all the data to input it. You'd have to know every single player on the field, and their health, their mental state, etc... plus the coordinators, field conditions, refs, you'd have to have so much data and I can assure you no sheet a coach is holding is going to have that.

So TLDR is that analytics is still at best an educated guess. Furthermore, it can trick someone into taking an unnecessary risk.

 

[teamplayer]

Analytics is just today's fancy word for playing the odds or using statistics. As we all know, statistics can be manipulated to say just about anything we want them to say. The old saying about there being three types of lies often comes to mind. There are three types of lies: lies, danged lies, and statistics.

 

[Tidelines]

If we had Jalen Hurts at QB and Landon Dickerson at guard . I would say run that Eagle's sneak everytime. Otherwise not a real big fan of doing it much.

 

[davefrat]

Seems that part of the problem is that when the stats say to go for it, and you go for and it doesn't work, then the stats tell you that it's more likely you'll get it the next time you try it in a similar situation and it can snowball like a gambling addict at a roulette table...the old "yeah, I know I've lost 10 in a row so it's GOTTA work this time."

 

[Bamabuzzard]

Yea that's definitely fair. But I think the larger analytics community is mostly trying to sell it as a slight edge

Honestly, I'm not as familiar with how analytics work in the football world as it does in the baseball world. I imagine there are a lot of similarities. I coach 10U, 13U, and 16U (HS level) and also know several D-1 coaches in the region who I talk to on a frequent basis. I can tell you that in baseball analytics (at a lot of levels) has gotten out of hand and too involved in the game. One of the D-1 coaches I communicate with started off with dyed-in-the-wool analytics but after years of using them, has started to back off the degree to which he uses them. He said, "I've learned there are aspects of in-game situations that can't be captured in a number or percentage".

 

[rolltide7854]

Seems that part of the problem is that when the stats say to go for it, and you go for and it doesn't work, then the stats tell you that it's more likely you'll get it the next time you try it in a similar situation and it can snowball like a gambling addict at a roulette table...the old "yeah, I know I've lost 10 in a row so it's GOTTA work this time."

Josh Heupel says hold my beer.

 

[RollTide_HTTR]

I'm not against analytics, but I am against doing things only with as 4Q alluded to, the law of big numbers in mind. If you have to win every single game, 90% success rate or what have you might not cut it. Also for the record I'm a big stats guy and always have been.

I don't really want to get into a long winded argument about this today. I don't have the energy for it.

But I'll respond to this one post and then we'll just have to agree to disagree.

Anyway, to give an example, let's say the data says if you go for it you have a 91% chance of winning and if you punt you have a 90% chance of winning. Well, analytics says go for it. Ok, but let's say you don't make it and suddenly your odds drop to 80% chance of winning. Sure if you did it 1000 times you might come out ahead (or the model might not have the right data for your scenario), but you aren't in those other 999 scenarios, you're in the one where you just doubled your chance to lose.

a couple things here. I don't know that we can just make up numbers because the whole point of analytics is the numbers being based on actual data. So this is just a made up situation that idk if it is even accurate thing that would occur. I doubt analytics would tell you to do something that increases your likelihood of success by 1% if you are successful while decreasing your likelihood of success by 10% if you fail. That just doesn't seem like a very analytical conclusion to make.

So the counter to over reliance on analytics is always looking at what the worst case scenario is. What's the worst thing that can happen here? For instance Alabama has the ball on the opponent's two yard line, analytics might be (just give an example, not saying that's the case) fine with throwing a pass, but Alabama did throw a pass and it was intercepted and Alabama lost the game because of it and lost a chance at a championship because of it.

That's not a counter to analytics when the whole point of analytics is that it takes into account all the possible scenarios. At least the good models do for the most part. Analytics DO consider what happens in worst case scenario. That's frankly the part of what Saban said that makes no sense. It considers what happens if you don't convert.

Also what we are talking about here largely is going for it on 4th down. Which is one of the better modelling we have. Pass vs run is way more up in the air but FWIW analytics tends to tell you to run more in the redzone. Again, Analytics DOES consider things like interceptions.

Also, no matter how good the model is, it can't possibly know everything. For instance would the model know Mac Jones was playing that game after Tua got hurt? Would it factor that into the analytics of whether or not it was wise to throw from the two yard line?

Sure the model can't know everything. But can a coach know everything? Of course not. Saying a model can't be perfect doesn't really mean much when the alternative is a human decision based on what? gut? Can gut know everything? Or maybe you'll say experience but the overall point still stands.

I've seen a few coaches turn close games into blowouts against Alabama because they kept going for it on fourth down and failing. I don't doubt analytics said go for it, but did the model really factor in it was the Alabama defense they were going against? In a generic game with two generic teams facing each other, it might be better to go for it on fourth, but against Alabama it might not be.

few things here. 1. Good models consider the teams playing not just generic teams. and also this is why i think its overall probably more effective in the NFL because there is more parody and things are more equal. Idk what data specific people are looking at. 2. The whole point is to win a game not lose a close game. sometimes taking risks mean it might not work out and you might lose by more. But who really cares? the point is to win. This is an extreme example to make a point but if you're down 7 with 1:00 left and you're at the 20 and its 4th down. You could kick a field goal and make the game closer but you're not going to win. Or you could go for it and yes the loss would be a bigger loss but kicking a field goal for a closer win isn't really the point.

How could you even use analytics as a playcaller when you have Milroe as your QB? You don't know if you're going to get good Milroe or bad Milroe, I have no idea how well he'll play next game and there's no model that can predict that. So if I'm looking at an analytics model, and trying to figure out what choice to make, how do I factor in the unpredictability of MIlroe?

By this logic how could you every call any play with Milroe as your QB? You don't know if you'll get good or bad Milroe then might as well never call a passing play? This doesn't make sense to me.

No model will ever be able to fully figure all the variables out, there's just no way to get all the data to input it. You'd have to know every single player on the field, and their health, their mental state, etc... plus the coordinators, field conditions, refs, you'd have to have so much data and I can assure you no sheet a coach is holding is going to have that.

So TLDR is that analytics is still at best an educated guess. Furthermore, it can trick someone into taking an unnecessary risk.

Agreed no model will ever be perfect but if perfect is what we are aiming for we just shouldn't use any information ever. Because no information is ever perfect. The human brain for example is known to be emotional and make decisions based on emotion in high pressure situations. That is why people often recommend having a financial advisor to manage your money. Individual investors often act emotionally and buy when the market it high and sell when the market is low. Which is obviously not a recipe for success.

Ultimately analytics are a tool to be used to HELP make decisions. I don't think anyone should go 100% with what a model says all the time. But I do think it can be an incredible useful tool. Does that mean sometimes you will lose a game by more points? almost definitely. But it also means that if used well I think they will help decrease the margin for error during other parts of the game.

There is also this weird assumption that kicking a field goal or punting has no risk but that's simply not true. If you punt of kick you are giving the other team the ball and with that there is a chance you don't see the ball again for example.

Anyway even Saban in that interview uses some analytics by asking how many points a turnover is worth in a game. I think he says 3.5 or something. Where does that number come from i wonder...

TLDR: No models aren't perfect, Yes sometimes things aren't going to go your way and you might even lose by more points than you would have. But a good model used correctly by a good coach will HELP you make the best decisions you can.

My ultimate point is that Saban is wrong in this interview. The models do consider what happens if you don't convert. That's not something it ignores.

 

[Tidewater]

One of the D-1 coaches I communicate with started off with dyed-in-the-wool analytics but after years of using them, has started to back off the degree to which he uses them. He said, "I've learned there are aspects of in-game situations that can't be captured in a number or percentage".

It is a human activity. The decider needs to get eye to eye contact with the guys on the field, look into their eyes and see what's there.
In the Army, good commanders do this all the time. Germans call it "fingerspitzengefühl," literally "finger tip feeling," but it means getting a feel for the situation, something an analytics table or situation map can never do.

 

[RollTide_HTTR]

Honestly, I'm not as familiar with how analytics work in the football world as it does in the baseball world. I imagine there are a lot of similarities. I coach 10U, 13U, and 16U (HS level) and also know several D-1 coaches in the region who I talk to on a frequent basis. I can tell you that in baseball analytics (at a lot of levels) has gotten out of hand and too involved in the game. One of the D-1 coaches I communicate with started off with dyed-in-the-wool analytics but after years of using them, has started to back off the degree to which he uses them. He said, "I've learned there are aspects of in-game situations that can't be captured in a number or percentage".

Seems like he's learning how to use them most effectively to me. I'm sure there are times when following analytical models completely isn't the best idea. But I think waving them off and not using them at all is also a bad idea.

 

[JDCrimson]

I think I can say with high degree of probability that sign stealing is a better advantage...

 

[lowend]

The component missing in "analytics" is the human component that exists in the moment and the individuality of "your" team.

You can't quantify the emotions and decision making capabilities of 18-22 year old men either.

 

[CrimsonBloodn62]

Honestly, I'm not as familiar with how analytics work in the football world as it does in the baseball world. I imagine there are a lot of similarities. I coach 10U, 13U, and 16U (HS level) and also know several D-1 coaches in the region who I talk to on a frequent basis. I can tell you that in baseball analytics (at a lot of levels) has gotten out of hand and too involved in the game. One of the D-1 coaches I communicate with started off with dyed-in-the-wool analytics but after years of using them, has started to back off the degree to which he uses them. He said, "I've learned there are aspects of in-game situations that can't be captured in a number or percentage".

Especially in baseball. When coaches see a left- or right-handed hitter coming up late in the games, they always try to bring in a certain left-handed pitcher to face left-handed batters.

 

[UAH]

Especially in baseball. When coaches see a left- or right-handed hitter coming up late in the games, they always try to bring in a certain left-handed pitcher to face left-handed batters.

That is more of a law of physics in baseball. It is more difficult for a left handed batter to see the ball out of the hand of a left handed pitcher and significantly easier for the LH pitcher to hit the outside portion of the plate. As an example Ted Williams hit around .225 against Whitey Ford.

 

[uafan4life]

Honestly, I'm not as familiar with how analytics work in the football world as it does in the baseball world. I imagine there are a lot of similarities. I coach 10U, 13U, and 16U (HS level) and also know several D-1 coaches in the region who I talk to on a frequent basis. I can tell you that in baseball analytics (at a lot of levels) has gotten out of hand and too involved in the game. One of the D-1 coaches I communicate with started off with dyed-in-the-wool analytics but after years of using them, has started to back off the degree to which he uses them. He said, "I've learned there are aspects of in-game situations that can't be captured in a number or percentage".

This is something I've tried to explain to several people in several contexts that many seem incapable of understanding:

Analytics are generally great but specifics always trump generalities.

And this is coming from a person who tends to be so logical that his wife sometimes calls him Sheldon...

One great example is in regard to gambling, specifically in video poker. Analytics were basically invented in gambling and the most ubiquitous usage, certainly by the public at large, is in video poker. However, there is a glaring weakness in the "games" of those who blindly follow the percentages in terms of expected value once the law of large numbers applies.

Sure, if you play video poker for a few hours every single day, the percentages will apply to you in the long run. However, the less often you play, the greater the disparity between the expected return and the actual return becomes.

Additionally, analytics is a purely historical context. When attempting to use any historical context as a predictor of future results, the accuracy of such a prediction is directly correlative to the percentage of common factors at play between the historical events and the future, or current, event(s).

For example, if you are playing Ultimate X Double Double Bonus 10-play and are dealt a QH, JH, 10H, QS, & 7D then the analytics will tell you to keep the QH, JH, & 10H to go for a straight flush or royal flush. However, if you were dealt a full house on the previous hand, now you have 12X multipliers on all 10 hands and you need to take that into account. And if, like most people who aren't professional gamblers, you're only likely to see this type of hand two or three more times in your whole life, then you also have to take that into account. You see, the analytics traditionally treats all hands as equal and expects you to see each hand thousands of times in your life. A more accurate analytical approach for this specific event would be to only compare past events where you were dealt Q-J-10 suited versus a pair of queens with 12X multipliers on 10 hands while playing Double Double Bonus. Additionally - for the average person - the expected value should be calculated based upon the result of playing this hand only two or three times rather than averaging a few thousand versions of this hand.

You see, if you only play this hand three times ever - the actual results are most likely (90%+) going to be losing money by following the standard analytics. However, going against the analytics and holding the pair of queens three times gives you around a 50% chance of making money.

So, the more specific the event the more specific the analytics needs to be to actually be an accurate predictor.

To translate this into a college football application...

First, you need to know the current, basic circumstances, e.g., the down, distance, and yard line.
Second, you need to know the most successful set of possible plays any team with your style of offense can execute for that situation.
Third, you need to know the subset of such plays that is most successful against the style of defense your opponent is playing.
Fourth, you need to know the subset of such plays that your current team can most successfully execute.
Fifth, you need to know the subset of such plays that your current opponent is least successful at defending.
Sixth, you need to evaluate that subset of plays against your current offensive disposition, such as if your starting left tackle left the game last quarter or two plays ago or whatever.
Seventh, you need to evaluate that subset of plays against your opponent's current defensive disposition.
Finally, you need to weigh the balance of the potential success of whatever set of plays is left, if any, against the likeliest results of either the failure of that play or else taking the safe route, such as punting, given the specific circumstances.

Typical analytics only tells you the odds of any random play-call being successful given the current, basic circumstances. It doesn't factor in what your team is good at executing, what the other team is good at defending, or any of the other six sets of considerations facing you in this situation. And it doesn't because it can't, not feasibly. The sample size of such specific situations would be so small that the data would be useless, even if it tried.

So, sure, for any random team that goes for it on their own 45 yard-line instead of punting the opponents' expected points on the ensuing drive might be historically expected to be nearly identical. And you could reasonably assume that means the decision is virtually immaterial and the potential upside of extending your drive is worth that minimal risk. However, if you've already gone for it on your forty-something yard-line and didn't make it four times this season and all four times the opposing team scored a TD on their ensuing drive then those specifics have to trump the generalties of the analytics.

That's because the accuracy of predictive analytics for any specific context is directly correlative to the percentage of common factors between the contexts of each and every data point in the analysis and the current context...

 

[Bamabuzzard]

Seems like he's learning how to use them most effectively to me. I'm sure there are times when following analytical models completely isn't the best idea. But I think waving them off and not using them at all is also a bad idea.

Baseball keeps too many stats for anyone who is involved in baseball to do that. I think, as you've stated, people are starting to find when it is appropriate (for their team) to go with the analytics and when not to. No team, regardless of sport, is built the same. The coach has to KNOW his team to be able to make the right decisions.

 

[KrAzY3]

I don't want to turn this isn't something that grows eternally either, so I'll just try to cherry pick my responses a bit. I just like stats a bit too much to not discuss them at all. So I certainly will take no offense if you don't reply further.

I doubt analytics would tell you to do something that increases your likelihood of success by 1% if you are successful while decreasing your likelihood of success by 10% if you fail. That just doesn't seem like a very analytical conclusion to make.

This is the sort of thing analytics might do if the chance of success is roughly ten times greater than the chance of failure (within the model).

That's not a counter to analytics when the whole point of analytics is that it takes into account all the possible scenarios. At least the good models do for the most part.

Analytics can't consider everything though, it might be able to consider all possible outcomes, but it can't consider the cause of all possible outcomes. For instance, is the field muddy? Does the quarterback have a hurt ankle? Both those things can change the play, but no model that I'm aware of takes those into account.

Sure the model can't know everything. But can a coach know everything? Of course not. Saying a model can't be perfect doesn't really mean much when the alternative is a human decision based on what?

This one I had in mind when I first came into the topic. The mind if basically a computer, one that's capable of processing a ton of information with incredible speed. We don't think of it like that usually for some reason, but every coach is doing a form of analytics it's just based on his observations and knowledge. The best ones could consistently come out ahead of a lot of analytics driven choices because they can take in more data in real time without even trying to and know for instance it might be a bad idea to go for it on fourth down and three because the way his quarterback's been playing, despite analytics saying it's a good idea.

Anyway even Saban in that interview uses some analytics by asking how many points a turnover is worth in a game. I think he says 3.5 or something.

A play can't cost you 3.5 points though, right? All turnovers are not created equal, where you turn it over, when, how your defense is playing, there are so very many variables. However, analytics give you this number that's so devoid of context. A lot of analytics can do that because they can average everything out to the point that it can't even come close to matching any particular scenario.

You have to start doing so much with that data in terms of setting up a scenario and considering it before you have a number that actually informs your choice. Analytics can easily give you this bland thing that seems like it tells you something when really it basically took peanuts and turned it into peanut butter.

 

[RollTide_HTTR]

This one I had in mind when I first came into the topic. The mind if basically a computer, one that's capable of processing a ton of information with incredible speed. We don't think of it like that usually for some reason, but every coach is doing a form of analytics it's just based on his observations and knowledge. The best ones could consistently come out ahead of a lot of analytics driven choices because they can take in more data in real time without even trying to and know for instance it might be a bad idea to go for it on fourth down and three because the way his quarterback's been playing, despite analytics saying it's a good idea.

There is a longer response to all of this but let's put it this way.

Just consider analytics another data point for the brain to process. Why wouldn't you want more information to help make the best decision?

My point about turnovers costing you 3.5 is that even Saban clearly uses some analytics in his thought process because he mentioned it in that interview.

 

[CajunCrimson]

We need a course on brevity