The Imminent Revolution in How Professional Hockey is Played and Coached

When Billy Beane changed the world of sports forever at the beginning of the 21st century by reinventing how a team chose to play the game of baseball at a very basic strategic level, he was blessed with decades of detailed data on which the statisticians he later elevated to legendary status were able to base their assumptions. While there have been significant advances since then in the amount and granularity of data collected in major league baseball, Beane walked into a baseball culture that amongst all other sports had always been maniacal about counting everything they could. Which made what Beane did all the more incredible, he simply dared to listed to what the data prescribed a baseball team should do to optimize winning games.

A few years later, basketball statisticians would dare to simply look at the data and found the game was being played in a strategically inefficient way. Mid-range jump shots had just about the same probability of going in relative to three-pointers, yet players and coaches were ignorantly wasting potential points. The percentage of three-point attempts taken between 2001 and 2018 went from 18% to 34% and is still climbing as coaches and GMs rework their strategies. In 2017 the Houston Rockets attempted 40% of their shots from beyond the line, and had the best offensive in basketball, which should have surprised no one.

While the majority of people like to focus the revolution of data science being used in sports on how to evaluate the players as individuals, which of them to draft, how much to pay them, and when to trade them, the true revolution has been in a better understanding of just what wins games. This narrative is equally tough for us to accept, for one, because sports at large are often the story of individual greatness, and two, because as fans, general managers, coaches, and players, we all would like to believe we know what wins games. The orthodoxy of “how things are just done” is only so strong against the very obvious correlation between team winning percentages and team revenue. Billy Beane may have simply wanted to win games for its own sake, but in today’s sports world owners are acutely aware that winning equals dollars.

Within the next 5 years, a similar revolution will sweep through the National Hockey League. Beginning in the 2019-2020 NHL season, an incredible new data set will be collected league-wide. RFID chips in every player’s should pad and every puck will transmit location, velocity and acceleration 2,000 times per second with inch level accuracy. In the book of hockey, the delineation between the dark ages and the time of enlightenment will undoubtedly be the release of player and puck tracking. We are about to learn what actually wins hockey games.

But before we get to how we’ll cross that chasm, which often produces violent upheavals, I want to briefly review why analytics have been slow to impact the game of hockey.

The randomness of a hockey game relative to other professional sports is incredibly high. The game moves fast to say the least, most plays are broken, and that little vulcanized piece of rubber, well, it’s rubber. If you attempted to account for the number of discrete categorizable actions taken by all of the players in a single game, which today would be nearly impossible, it would likely be several orders of magnitude larger than any other major sport. Baseball has a limited number of discrete plays involving a limited number of players in each moving in mostly straight lines. While football operates in a similar fashion, we increase the data set by including a larger number of players in each play along with the potential options of their activity and direction, as well as the combinations and permutations of the group’s activities. Like football, a basketball game is built on a set of mostly discrete possessions, yet with more randomness in terms of turnovers. The players are standing still most of the time, or simply running in a straight line. But hockey amongst them all is the most fluid, most error-prone, most random. It is this randomness, multiplied by the incredible speed of today’s game, and the threat of injury from vicious collisions that make hockey the most exciting game in the world. It also makes collecting data really really hard.

It’s not as if we have nothing today though. Some teams have spent the time to look back at game tape to identify some of the variables beyond the box score that matter to both individual performance and team play. These data sets are dependent mostly on humans analyzing film to identify a limited set of variables. It’s arduous, and certainly not real time in any way. What’s more, the data set is somewhat anecdotal in its measurement.

The vanguard of advanced hockey analytics is what hockey stats nerds call Score-Adjusted FENWICK. The basic premise being that there is a decently strong correlation between goal differential and how many unblocked shot attempts a team makes (adjusted for the quality of those shots based on their location) relative to the other team, taking into consideration the score of the game (teams tend to employ different strategies relative to the score being more or less aggressive offensively).

While the concept of taking more shots from high probability areas of the ice relative to your opponent seems like a strong a-priori economic hypothesis, the real issue is, how do we make that happen? This is where hockey significantly deviates from the other major sports, especially baseball and basketball. The manager’s decision to bunt versus let the batter swing to move a runner over or attempt to drive him in from 2nd base is a discrete play and decision. When a basketball player chooses to shoot from beyond the three-point line instead of driving to the net, they are taking into consideration the defense presented to them sure, but getting to the point of being able to make that decision which leads to a scoring opportunity is handed to them with little or no objection by the opposing team. The success of those attempts will be impacted by the defense of course, but the number and location of those attempts is largely up to the offensive player.

Some periods of a professional hockey game may feature a team directing fewer than 5 shots towards the net while others may be 10 times that. The dispersion in shot attempts is also significant. With such a wide standard deviation in this critical metric relative to basketball or baseball, we have to ask, how does a team produce more and more high quality shot attempts? Given the standard deviation, this variable can’t possibly be a simple function of whether hockey players decide to pull the trigger or not as it is in basketball.

I want to walk you through just some of the important activities that we are going to be able to measure with precision through analysis of the new player and puck tracking data set that will help us answer that question, and many others. This analysis will require an approach that looks a lot like a hybrid between how machine learning systems are able to identify the difference between a dog and a cat, and classic financial quantitative research into equity factor models. It will begin with the need for humans to identify, using film matched up in time to the data set, examples of each specific variable we are attempting to measure. So for example, we need a bunch of examples of what it means when a defenseman pinches in from the point to keep a puck in the offensive zone. This may seem simple on the surface, but we need to give our supervised machine learning system which looks at the tracking data enough examples in order that it is able to go through the entire data set and find all of the others with high precision, or identify new ones in real time. This will take place for every discrete activity we believe could have an impact on the game. As a side note, a significant advantage will accrue to the NHL team which chooses to spend the time to do this work, because it will be the basis for everything else done with this massive data set, and it’s not easy. Without this, you have a whole lot of very detailed but meaningless data.

Let’s take a look at some of those variables before we get into the factor model building we can do once we have them all sorted and measured.

Defencemen

• What is his gap control on the rush, is it tight or loose, what is the standard deviation of this variable, is the player consistent regarding their ability to defend the rush tightly?
• What percentage of the time are they able to break up the rush by separating the offensive player from the puck or causing a broken play via their defensive positioning? Same for odd man rushes.
• How effective is he at net front checking, which we could define as the number of scoring chances against from the slot or crease area emanating from the offensive player he is checking (closest to).
• How many odd-man rushes does the team surrender with the player caught up ice?
• How often does he skate the puck out of the defensive zone effectively?
• What percentage of the time does the player’s breakout pass lead to a zone exit? How often does it lead to a turnover in the defensive zone?
• What percentage of his stretch passes are completed versus lead to turnovers?
• What percentage of the time does a player effectively pinch in the offensive zone? How often does his pinch lead to an odd-man rush against?
• What percentage of his shots from the point get through to the net?
• How effective is he at creating scoring chances when moving down from the point?

Forwards

• How often does a forward create a zone exit when receiving a puck on the half boards in the defensive zone? How often does he turn it over?
• How effective is the player at creating offensive zone entries by carrying the puck across the blueline?
• How often does the player give the puck away at the blueline attempting to enter the zone?
• How often does the player “punish the defenseman” after the puck is dumped into the offensive zone corner?
• How often does the player take the puck to the net to create scoring chances between the dots (aka high traffic areas)?
• Effectiveness at breaking up plays on the backcheck?
• How hard is the player backchecking?

General

• Percentage of passes made which lead to odd-man rushes for?
• Percentage of giveaways leading to odd-man rushes against?
• Number of times a player touches the puck?
• Total puck possession time?
• Average time between puck possession and distribution (“holding onto the puck”)?
• Pass completion percentage?
• Skating speed with the puck?
• Shot velocity?
• Line change laziness (speaks for itself)?
• How often does a player get involved in after the whistle extracurricular activity?
• Number of shots blocked or caused to miss the net by way of being “in the lane”?
• How physical is this player (this one will be tougher but I have a feeling we’ll be able to figure out the mix of speed, acceleration, and proximity between opposing players to get this right)
• How often does a player “stop-on-the-puck”? Often a gripe of coaches against players who aren’t as involved in the play as they would like in order to battle for pucks.
• Shoot vs pass % on odd man rushes?
• Distance skated is interesting, but with location, speed and acceleration we can produce a statistic for “effort” that will likely be more relevant.

This is just a fraction, though maybe the more important fraction, of the list of variables we will be able to collect in real time for each player. There are strong hypothesis for why each of these variables may be important to understanding how a player contributes individually to the success of his team on the ice. It’s almost certain that some of these variables will be highly correlated with others to form what we believe to be certain “types” of players. The “power forward”, the “stay at home defenseman”, the defenseman who is a “riverboat gambler”, the forward who is a “grinder” or the “sniper”. But at least we will have hard evidence for placing players into these categories, or others that basic cluster detection models show us have been hiding in plain sight all these years. Maybe we’ll be surprised about what the data says regarding what some of our favorite players are actually doing on the ice compared to what we think they are doing out there. It’s very likely as well that as players age their games change and the popular sentiment about what role they play on the ice or should be playing is misguided. It’s also likely that we’re able to identify the optimal mix of these “roles” on a team, or on a line.

The difference now will be that instead of simply asking what leads to a higher expected goals ratio when a player is on the ice, we’ll be able to see how each of these variables correlates with the outcome of the game. It isn’t always the case that the goal of each shift for a hockey player is to score a goal and prevent the other team from doing so. Yes, on the surface this is what hockey boils down to, but on the bench and in the locker room, everyone knows that the variable some players and some lines are optimizing for in their play is not outscoring the other team. This is best explained through the example of a long playoff series where we intuitively believe that if a team dumps the puck into the offensive zone enough, forcing the opposing defense to turn and go back for it, opening themselves up to get hit again and again and again, this will eventually pay dividends later in the series as those players get tired, weary, and possibly injured enough to make mistakes, leading to offensive zone turnovers, and thus goals. If you watched this year’s NHL playoffs you undoubtedly remember Dougie Hamilton allowing  Alexander Ovechkin free access to the puck in this scenario after peering over his shoulder at number 8 barreling down on him and basically saying, I don’t really feel like getting obliterated here. That play, which lead to a goal a few seconds later was the result of several games of Hamilton getting abused by Washington forwards. Our statistic for forwards who “punish the defenseman” the most could potentially be the most highly correlated variable to winning a playoff series (though it didn’t work out for Washington in this case), where in the regular season it’s a strategy that should be shelved given that we don’t care if tomorrow night that defenseman is tired or injured facing a different team.

The next level of our analysis will roll up from individual player stats to team statistics and things that coaches can specifically make decisions to do, or not do. For example, different coaches will employ different forechecking schemes, at different times during a game depending on the score, and even for different lines. Our new data set allows us to easily see what these schemes look like, and when they are successful or not. When the Tampa Bay Lightning won the Stanley Cup in 2004, their head coach John Tortorella had a saying about their forecheck that “safe is death”. This meant basically, go after the puck at all costs, all the time, with three forwards in deep. Other winning coaches, or in this case GMs have been notorious for telling their players to sit back and use the trap, frustrating opposing teams into poor decisions. Current New York Islanders GM Lou Lamoriello is famous for this strategy having produced 5 Stanley Cup finals appearances in 7 years with the New Jersey Devils, winning 3 of those. After joining the Islanders this past year the team went from worst in the league to best defensive team while advancing to the 2nd round of the playoffs after being a consensus lottery team at the beginning of the season.

This is to say, whether it’s true or not, the hockey world believes that forecheck strategy matters, a lot. We will be able to measure how different forecheck strategies impact other individual variables from the other team as well as our own ability to produce quality shots or simply the correlation to goal differential. Let’s take a look at some of the other team play oriented variables we should be able to collect.

Team Play

• How often do we recover the puck after dumping it in?
• How often do we successfully carry the puck over the blueline?
• Shot location (outside vs inside)
• Shot setup (off the rush, pass across the slot, pass from behind the net, rebound, etc.)
• Time spent with possession
• Time spent with possession in the offensive zone
• Effectiveness of certain forecheck schemes

All of this analysis must include rigorous in and out of sample testing. It is possible that the number of observations associated with some of these variables is simply not great enough to give us a good confidence interval in the persistence of one player being better than another, or one team strategy being optimal. It may be that for individual players we need several years worth of data to get a strong confidence level for their play, while for rolled up team play our confidence comes much quicker.

So now that we have organized all of the individual and team variables that we believe could potentially be indicative of success and failure, we’ll start connecting how one leads to another. Eventually what needs to be produced are four separate “deliverables” to the GM and coaches.

1. The software infrastructure necessary to store and handle this amount of data isn’t simple,  but it also isn’t rocket science. Other domains deal with similarly large but highly structured data sets. You won’t want a third party firm handling this for you because your ability to analyze the data will decrease significantly. I’m not sure exactly how the NHL is going to be making this data available to teams, but the hope would be that they provide real time streaming APIs with endpoints for each player and puck on the three variables of location, speed and acceleration. It would also be useful if they provided a REST API for historical data that could be queried at some interval, like once a day. In either case, we are going to need to store all of this data ourselves, along with all of the derivative features we develop from the data which should also be close to real time. If the NHL releases data on all teams to each and every team, which would be incredibly useful, multiply the amount of data we need to store by 32 come the inclusion of the Seattle team a few years from now.

2. All of the individual player statistics, as well as team strategy statistics, are going to be made available. Specifically teams are going to need a real time in game dashboard that shows absolute numbers along with ratios for certain variables relative to historical trend. Players should absolutely be graded on a curve. If a specific forward normally backchecks at a certain speed relative to his teammates, we want to understand how he’s performing tonight relative to that. If the team as a whole normally backchecks at a certain speed, we want to know what’s going on relative to that as well. Maybe the team is “playing guilty” tonight and needs a kick in the ass. Giving the coach the ability to assess what’s actually taking place amongst his players instead of having them exclusively use the eye test should improve their basic ability to coach and motivate players to do specific things. Players might feel certain criticism is unfair today because coaches are acting upon anecdotal information, but you can’t hide from these numbers. If the team is running into a brick wall at the blueline tonight and turning the puck over, the coach will know it and be able to change strategy. This dashboard must also be able to highlight off trend or abnormal performance so that coaches don’t have to sort through it by hand. I envision a new coaching role that’s something like an offensive or defensive coordinator in football, but more data-driven.

3. General managers need to understand how specific variables actually add up to how teams win games and which players possess better stats for those variables. We should be able as well to figure out over a longer horizon what the right mix of variables is needed on a team. If a team doesn’t possess a player or players with needed variables, the GM may need to target those in trade or free agency. Every year trade deadline deals are struck to bring “grit” or “toughness” to a team’s 3rd or 4th line. But at least half of those trades are failures. Can we improve that percentage, most likely. While the NHL and NHLPA have struck a tacit agreement that GMs are not allowed to use the tracking data in salary arbitration hearings or contract negotiations, it’s obvious that this data will seep into that process even if it’s not explicitly used in the meetings. No GM is going to ignore it, and shouldn’t. Which is also why the NHLPA should be allowed access to the data as well and provide players with their own version of these stats.

4. The ultimate goal here is to understand what activities actually win games and whether strategy itself needs to be adjusted to take advantage of our increased understanding. The evolution of basketball didn’t go from, “teams that shoot more threes win more games" to, "let’s all shoot more threes.” No, they first had to realize that the shot pattern of the players itself was inefficient as it related to expected points, and then they were able to prescribe the solution of shooting more threes as a function of winning more games once they had the data (see Golden State Warriors). This research will be delivered less as a dashboard and more as strategy papers over time. Bill James didn’t reinvent baseball overnight, it took years, and then decades for anyone to pay attention to him after the data prescribed obvious actions for teams to take.

Which leads me to why all of this is going to be hard.

I want to draw a corollary to what’s happening today in the world of active asset management. The buzz word everyone uses is “quantamental”, representing the combination of fundamental analysis/data with factor model building. Unlike in finance, we can’t hand the coaching or GM roles over to a set of algorithms, nor should we. Yes, quantitative systematic strategies have grown significantly in AUM, especially on the passive side where it’s beta and not alpha we’re looking to gain exposure to. But on the active management side, the capacity for these strategies will ultimately be limited. The promise of quantamental investing is in the ability to leverage what we would call “the eye test” in hockey, as an input to our quantitative model. Humans are in fact great pattern matchers and can project future outcomes, especially when nonlinear, better than extrapolative algorithms, but only when they are forced to hand over their projections under the right heuristics and in the right structure at the right interval. Yea, it’s hard, which is why most quantamental strategies have failed. Not because the theory itself is unsound, but because the ability to force or cajole humans into providing the data or listening to the algorithmic recommendations from that data is limited. Portfolio managers don’t want to be reduced to cogs in a wheel, they have massive egos, and desire to make portfolio decisions. The fact that they are suboptimal tools of execution of their own good ideas is irrelevant to them, even when millions or billions of dollars they could be making is on the line.

The success or failure of this entire endeavor will be dependent on the ability for the data team to both produce end products that are usable by management, and bridge the chasm between the eye test and the data. Humans will not listen to statistics in a vacuum, but they will change behavior if you can draw a narrative for them that they understand which is supported vigorously by the statistics which alone they would have dismissed. A successful approach to this will look very staged. While the data team may produce all of the work outlined above, its inclusion into team decision-making process is going to be gradual starting with a limited set of individual player statistics, allowing coaches to use it as they wish. It’s likely that some players will then come directly to the data team, as happened in baseball. The inclusion of a dashboard for coaches will come next and require a significant amount of trust and hand holding between the data team and coaches. The data team will have to compromise on a regular basis regarding what variables are front and center at the start even if they believe those are the wrong variables. Mistakes and learning will have to be made because it will be impossible to force coaches and GMs to be efficient users of these models from the drop. Over time efficiency and efficacy will increase. Last, coaches and GMs will shift their strategy for how they play the game and acquire talent relative to what these models say, and only by seeing success on an individual player level. I don’t expect we’re going to see any Billy Beanes out there willing to go all in all at once on every level.

Or maybe we will.