What Failure Rates Say About NHL Goalie Drafting
It is conventional wisdom that goaltenders are so hard to project that NHL teams are far worse at drafting goalies than other positions. In a recent article on the difficulties of evaluating goalies, for instance, ESPN’s Corey Pronman stated, “once they get closer to (or reach) the NHL level, the evaluation becomes splintered. At that point, there is a much higher failure rate for top prospects among goaltenders than for skaters.”
This isn’t quite true, although it depends greatly on how you define the terms “top prospects” and “failure.” This wasn’t Pronman’s overall point, so he didn’t offer any indications of how he came to that conclusion, but using a reasonable definition of both “failure at or close to the NHL level” and “top prospects” shows that the failure rate for goaltenders is largely comparable to that of skaters.
Essentially, what the draft data shows is that NHL teams are generally bad at drafting at all positions once you reach the bottom of the first round. Only among the very obvious top prospects do failure rates indicate anything like efficiency.
When analyzing draft outcomes, we have to account for the structural differences in drafting and career paths between skaters and goalies. There are three basic issues to remember. Goalies are drafted later than skaters, there are fewer goalies drafted, and goalies have fewer opportunities to get ice time, especially early in their careers.
For instance, among the 360 prospects taken in the first round of the NHL draft between 2002 and 2013, 61% were forwards (221) and another 33% were defensemen (119). Only 20 goalies were drafted in the first round in those eleven years. That’s 5.5% of first round picks. There were no first round goalie picks at all in 2007, 2009, 2011, or 2013. Comparisons between first round goaltenders and first round skaters must be done with this in mind. (All data is from hockeydb.com.)
To an extent this reflects the makeup of NHL rosters, where goalies account for less than 10% of a 23-man squad. The structure of rosters does, however, undeniably make it harder for young goaltenders to actually accumulate measures of value provided. A significant part of many goalies’ careers is to provide value in contingency–relief and emergency appearances.
All of this means that we cannot compare draft position to draft position or ice time to ice time between skaters and goaltenders and accurately reflect who is failing and who is succeeding. The benchmarks used must be appropriate to each situation.
Taking again the drafts between 2002 and 2013, we can calculate a simple measure of the value provided to a team by dividing the number of NHL games a player has played in by the number of years since they were drafted (GP/y). Thus a player who enters the NHL young and stays in the lineup will have a higher score than someone who enters late or who never sticks. Someone drafted 5 years ago with 5 games will have a score of 1. Someone drafted 5 years ago with 200 games will have a score of 40.
Because it’s much easier for a skater to accumulate games than a goaltender, what counts as failure or success also varies. For skaters, a GP/y score of 30 generally indicates a marginal NHL player–perhaps not a success but not a failure, either. For goalies, that number is closer to 5 GP/y.
The proportions of draftees with no games played (a very reasonable definition of a failure rate) and those who are marginal (30GP/y for skaters, 5 GP/y for goalies) are remarkably similar, even among top prospects. The sole exception is the rate of goaltenders from the first round classified as marginal.
|Round 1||Round 2||Rounds 1 & 2|
First round failure rates are worth a closer look. While it appears that skaters taken early in the draft are more likely to be better than marginal players, the vast majority of that advantage comes from players drafted in the top 15 spots alone and it is largely a function of who gets drafted first.
In essence the top 15 draft picks are highly successful and are almost never goaltenders. When goalies do get drafted there, they are as likely to enter the NHL as forwards and defensemen and are as likely to be better than marginal. There are simply not very many of them. Thus we get a very high success rate at the top of the first round that doesn’t accrue to goaltenders.
So what does this all mean?
In essence it means that NHL teams are generally bad at drafting at all positions with the important exception of the very, very best. Those players who are exceptionally outstanding at age 18–the can’t miss prospects–have a very high rate of success. That efficiency is largely gone by the second round, where more than 40% of all draftees will never play in the NHL at all and around 70% or more will become only marginal players.
NHL teams are bad at projecting players at all positions, not just goaltending and goalies don’t really fail at much higher rates than skaters.
The question of how to improve goaltender evaluation remains open. There are still no good, objective statistical measures that predict future success with any degree of accuracy or usefulness. There are several efforts to improve these statistics and in time those might bear fruit.
In the meantime, the difference between the failure rates of European goaltenders and Canadian goatenders indicates that greater exposure, especially by well-trained goaltender scouts, leads to better outcomes. It isn’t that NHL teams can’t scout well. It’s that they don’t scout enough.
More importantly, teams need to invest in goaltender development and to develop an overarching, systemic approach to the integration of goaltending and team defense. Find the players who fit the style of play for your team and develop them accordingly rather than leaving development and fit up to chance.
The team that successfully identifies how to match goaltending and team defense and that locates and retains goaltenders who fit into their system will have found a competitve advantage that is hard for others to imitate.