Last week it was announced that the New Jersey Devils had hired Sunny Mehta to be their Director of Hockey Analytics.  Just over a month ago, Craig Custace wrote a piece($) for ESPN Insider on the New Jersey Devils search for a Director of Hockey Analytics.  In that article, there was an underlying tone that hockey analytics was expensive.  Ryan Lambert of Puck Daddy nailed a part of this a couple days after when he wrote (under #3):

I love this idea that more NHL teams aren’t investing in analytics because the price tag reaches into the hundreds of thousands of dollars. This is a league in which a team still employs Colton Orr for close to a million dollars per season.

Maybe teams would rather not spend $250,000 to find out that they shouldn’t pay guys like him $925,000.

So this got me thinking about whether we could estimate the amount of return that spending $250k on analytics would yield.  I decided to do a bit of back of the envelope computations to estimate what would be the benefit from having an analytics staff.  No new analyses will be presented here, just some crude approximations.  Almost all of this is low hanging fruit and no doubt there are improvements that can be made. The list is not comprehensive.  I simply want to make the point that to say that hockey analytics have not proven their worth is ridiculous and I don’t need to be too thorough to do that.

I’ve broken down what follows into three categories: drafting, strategy and player acquisition.  Within each category, I’ve estimated the return via goals, or points or wins.  Following some previous work by Gabe Desjardins, I’m using that a win is worth 2 points and is worth roughly $2MM.  Consequently, a point is worth $1MM.  (Note the new CBA prompted me to round down slightly but that rounding should likely go up given what is expected for  the salary cap next year.)   We’re also going to look at things on a yearly basis so you might only get to sign a free agent goalie every fourth year so we’ll take 1/4 of the value from that in this assessment.

Drafting

The first area we’ll check  is drafting.  There are two aspects to this: evaluating players to draft and determining the value of draft picks.  The former is a good bit more difficult to evaluate since it is harder to get a sense of how much can be gained in this regard.  I also think analytics can help in this area both in predicting the future and in evaluating mistakes of the past. 

Some recent work by Eric Tulsky is suggestive of the data that might be mined from leagues outside the NHL and might be useful for projection of player performance in the NHL. This work helps to determine how a player is being used by their current team and how difficult is the quality of opponents with whom they share the ice. Such supplemental information might yield additional information that would be useful. If a players is generating 25 goals in 75 games but is playing sheltered minutes then we may want to draft others who are producing . This work is relatively new and so it is harder to evaluate how they will perform. However, it is likely safe to say that such methods are worth a couple of goals a year. Let’s say 2. That’s a 1/3 of a win or about $670k.

One area of drafting that has seen a good deal of work is that idea of league equivalencies. The idea of these approaches is to look at the performance of a player who spent year 1 in a non-NHL league (e.g SM-Liiga in Finland) and year 2 in the NHL. By looking at the relationship between points in these two years, we can create a measure of the quality of the non-NHL league but also a method for estimation of  production once a players has joined the NHL. The most famous example of the application of this sort of work comes from the decision by the Winnipeg Jets in 2011 to draft Mark Scheifele over Sean Couturier. The projections for Couturier were clearly higher than those for Scheifele as was noted at the time. Couturier has gone on to perform very well for the Flyers and is probably worth a couple of wins a year to them. Such circumstances are not going to crop up every year but when they do they are valuable. I’ll assume that such circumstances occur once every 4 years. Using Hockey-Reference.com’s Point Shares metric, the value of Couturier has been about 2.7 points over Scheifele over three years.  That’s about 0.9 points per year.  Or about 0.9/4 = 0.225 points or $225k per year.  

So conservatively the total for drafting is about $895k per year from drafting.  This excludes something like the creation and use of a value pick chart, for example this one or these.

Strategy

There is a good deal of evidence that pulling the goalie more often will yield some results. David Beaudoin and Tim Schwartz have done a nice study that suggests that teams should pull their goalies more often.  This mathematical approach uses goal scoring rates under a variety of circumstances and uses these rates to estimate the average number of points under a variety of circumstances.  They conclude by suggesting that on average a point per season can be gained with this approach.    As with any strategic innovation, it’s success will lead to (being copied and eventually) its demise.  Pulling the goalie faces many of the same difficulties as going for fourth downs in the NFL.  It is less likely to be adapted because the costs.  Nonetheless it is clear that analytics has suggested an improved way to play the game.  Assuming such an advantage would be adopted by other teams we might lose our advantage after a year and so we’ll round this down to $100K in value.  I do this here not because I don’t think it will work — I’ve seen first hand how well it works as my collaborator Chris Wells has led the NCAA in minutes with the goalie pulled for several consecutive years — but rather since I think it is less likely to be adopted by coaches.  

Another strategic move that has been shown to have a strategic impact is carrying the puck more and dump-and-chase-ing less.  Eric Tulsky has gotten a good deal of attention as the lead researcher in this area.  His paper on this was a poster at last year’s MIT Sloan Sports Analytics Conference.  The idea here is that carrying the puck into the offensive zone leads to better possessions which leads to more goals.  It is hard to get a sense of the value here but let’s try.  We can estimate about 50 zone entries at 5v5 per game (source here and here).  If we can increase the number of shots using this method by 0.5 per game that would roughly be 3 goals per year or about $ 1MM.  As above we spread out this gain over several years to get $250k per year.

So the number here suggest that it is possible to get $350k per year in value.  Value from the analyses given here is the most likely to be transient but it is also possible that analysts will find new areas to exploit.  

Trades and Free Agency

 Next we’ll look at player acquisition through trades or free agency.  One of the most useful aspects of analytics has been noted by Phil Birnbaum which is that they prevent stupid.  As the article argues, there is more to be gained by not being stupid than there is by being smart.  That is useful and a big part of analytics.  

The litany of large NHL contracts that might have appropriately received another look if analytics are involved is long and includes: Bryzgalov’s signing by Philadelphia, Douglas Murray’s trade acquisition by Pittsburgh from San Jose, or his signing with Montreal, or trading for Robyn Regehr.  Not every team signs or tries to sign a big free agent every year.  Let’s say one every four years that overplays by $1MM per year.  That’s a savings of $250k per year.  And we haven’t even looked at the contracts of Pekka Rinne, or David Clarkson, or Brad Richards, or the aforementioned Colton Orr.  As the salary cap grows this is likely to be an even bigger value.  

Summary

 There are lots of ways that hockey analytics can impact a team.  I’m an analytics guy so I’ve tried to be conservative here about the value that analytics can bring.  Overselling analytics won’t help it’s adoption.  The cost of a reasonable analytics department might be around $300k.  You could get a good deal of consulting input alone for $100k.  I think it’s pretty easy to see that analytics could yield (at a minimum) benefits for an NHL team of over $1MM from the areas listed above.  As I stated previously, these are estimates that are low and they are not exhaustive.  Hopefully from this, it should pretty clear that cost is not one of the factors holding back analytics in the NHL. 

Addendum

My friend Steve Argeris suggested the following:  I’d think about it a different way.  What are you spending your money on?  Presumably you can just have your cap guy or some $50k per year guy just stay up on what’s on the blogs and use the public websites reasonably well, so that’s really your baseline, not “nothing at all.”  So say to hire a PhD in statistics (ahem) and move him from his tenure-track position to a more expensive city, subscribe to various data sets, and pay freelancers (analysts, coders, database guys) $200k per year.  For that amount of money, what would it take every year to just break even?

 It would take:

1) not blowing one draft pick in the first three rounds every three years (merely moving one pick in three years from “never playing an nhl game” to playing, say, 60 games at a minimum salary OR

2) finding a league minimum guy that can do what a $900k guy can do once per year OR

3) finding a $1M player who can do what a $2M player can do every three years OR

4) recognizing a tactical or strategic decision that improves a team by three marginal goals over three years ONCE  

 In other words, if ANY of those three things happen, you’ve paid for yourself.  The likelihood of ANY of those things happening is pretty close to 100%.  The likelihood of more substantial benefit is far far higher.