Does High Danger Workload Affect Save Percentage?
Recently, I talked about the ways that shot volume affects save percentage. I concluded that facing more shots does mean slightly higher save percentages, especially at the game level, but that it wasn’t related to goaltender performance. Rather it was a relic of the way that percentages work as numbers. In other words, the math masks any changes in performance as shots increase.
This is an important observation because one of the standard explanations for save percentage increases in high shot games and seasons has been that the increase in shot quantity comes at the expense of shot quality. In other words, the explanation given has been that goalies who see more shots see more low quality, easier to stop shots. For instance, Matt Cane proposed this in 2013 when examining this question:
I have a few theories as to why we might see this. One possible explanation is that there are some score effects creeping in and the high shots against games are coming when the winning team is sitting back and allowing many low percentage shots from the outside.
This idea has taken root. These days it is often accepted as a given that for goalies as for teams more shots means more Low Danger (LD) shots and that a lot of Low Danger shots inflate save percentage. Conversely, more High Danger (HD) shots shots would deflate save percentage because they are “harder to stop.” (For an explanation of High and Low Danger shots, see War On Ice’s “Adjusted Save Percentage: Taking into Account High, Medium, and Low Probability Shots.“)
I couldn’t find any testing of this theory online, merely an assumption that this is an appropriate explanation. So I tested it.
There is very little indication that goalies who face more shots actually face more shots from the Low Danger zone or fewer from the High Danger zone or that goalies who face more High Danger shots put up lower save percentages relative to other workloads.
At the game level, the proportion of LD shots is essentially symmetrical around an average of 45%, but there is an extremely wide range of results. Tellingly, the more shots a goalie faces in a game, the closer the proportion of shots is likely to be to the average.
The same is true for the proportion of HD shots in individual games, although they are clustered around the 30% mark.
In other words, the more shots a goalie sees in a game, the less likely they are to see an extremely easy or difficult workload in these terms. Facing more shots in a game does not always mean facing more easy shots.
Looking at whole seasons, this becomes even clearer. More shots simply means more shots, not more shots of any particular kind.
Furthermore, even if a goalie did see more HD shots or more LD shots, that workload doesn’t have much of a predictable impact on save percentage, either by game or by season.
While the effect of workload is slightly stronger at the season level than at the game level, it is still almost nonexistent in any practical sense. This R^2 of .11 for LD workload was the strongest I found, and that is a pretty weak correlation with an extremely wide distribution. The effect of HD workload is even weaker.
In short, the proportion of HD shots or of LD shots explains very little about 5v5 save percentage. Players seeing more LD shots shouldn’t be expected to have higher save percentages. Players seeing more HD shots aren’t getting “dinged,” at least not across the board.
Compare this, however, to the importance of saving those HD shots that you do see. A clear rise, a tighter grouping, a very high correlation. HD save percentage has much more explanatory power for a goalie’s save percentage than HD workload does.
On the whole, location-adjusted save percentages don’t work the way they are getting used. They not only don’t they give a truer picture of performance, but they distract from those things that do appear to give information: 5v5 save percentage and 5v5 HD save percentage.
All in all we are left with a few conclusions:
- Save Percentage tends to rise a little in small samples as shots against rise, but that is largely because of the mathematical limits of save percentage itself. The rise continues to be somewhat visible in larger samples like seasons, but it is not strong enough to make us expect a higher save percentage in high shot seasons.
- More shots does not mean more Low Danger shots and more Low Danger shots does not account for the observed rise in save percentage, either at the game or season level.
- The proportion of Low and High Danger shots a goalie sees explains almost nothing about his save percentage, at least as those categories are currently measured.
- Goalies who see more HD shots do not get “dinged” much in their overall 5v5 save percentage and goalies who see more LD shots do not get rewarded much. As weak as it an effect as workload has, it is strongest for LD shots in very large samples.
- There is currently no way to determine what kind of save percentage we should expect from a goaltender based on the kind of shots he sees. That is: no baseline makes much objective sense given the current ways of measuring difficulty. This has the most impact on War on Ice’s Adjusted Save Percentage, but has implications for some of the various Expected Save Percentage formulas as well. These statistics should be used with great caution.
- On the other hand a goalie’s High Danger Save Percentage explains a great deal about his 5v5 save percentage and gives far more information than workload does and may be the most important measurement outside of 5v5 Save Percentage in evaluating goaltenders.
At this point, we really have very little understanding of how to measure goaltending performance well. While “shot quality” matters — otherwise, HD save percentage wouldn’t be so important to overall save percentage — our ways of measuring shot quality are currently inadequate. There is simply little evaluative value to location adjustments to save percentage as they are currently drawn up. They don’t clarify things much at all and they probably won’t as long as we are dependent almost exclusively on X-Y coordinates entered by hand in real time with little other information about shots.
In order to “see” goaltending better, we will need to break shot quality down and build it back up again.