David Hutchison | Jan 22, 2019 | 0
Does Shot Volume Affect Save Percentage?
Does shot volume affect save percentage?
This is a question that keeps coming up, and it usually means “do goalies perform better when seeing a lot of shots?” or “should we give extra credit to players who face a lot of shots for performing well under pressure?” In other words, does shot volume affect performance in a measurable way?
The only way we have to answer this with any objectivity is to look at save percentage. Does save percentage go up when goalies face more shots?
There have been multiple attempts to study this statistically over the years and the answer is mixed. There is some evidence that shot volume or rate has almost no relationship to save percentage. But if you bin games by shot volume (that is, put all games with, say, 15-20 shots together, all games with 20-25 shots together, etc.), a pattern does emerge in which more shots equals higher numbers.
So what’s happening here? Does shot volume affect save percentage or not?
The answer is actually “yes, save percentage rises as shot volume rises.” But in this case, the why is the most important part of the answer.
Save percentage increases as shots increase because of math, mostly irrespective of goaltender performance. In hockey, we see a limited and basically fixed range of goals allowed per game and a larger but also limited range of shots faced. Thus, as shot volume increases, goals allowed become a smaller and smaller proportion of the whole.
The possible values for save percentage are predetermined by this mathematical fact. Since 2010, the maximum number of goals scored at 5v5 in any given game is 8. The maximum number of shots faced at 5v5 is 51. And 12/20 is always going to be a lower number than 32/40. At the same time, 19/20 is always going to be lower than 39/40. Both the top and the bottom of the possible values of save percentage are higher when a goalie sees more shots.
At a game to game level, for instance, the range of possible values of save percentage for all non-shutout games looks like this (10-60 shots against, 1 to 8 goals against). This is it. There are no other possible outcomes.
As shots against increase, the range of the possible values of save percentage gets higher and narrower. Mathematically, if a goalie sees 10 shots against (if he doesn’t get a shutout) the highest his save percentage can be is .900. If he sees 20 shots against, he can earn at most a .950. If he sees 40, however, he can earn up to a .975. At 40 shots against, the lowest he can earn is .800. At 40 shots, a goalie can let in 8 goals for the same cost as 2 goals on 10 shots.
But what happens in real life? Where do save percentages actually land? I took the 5v5 results of all NHL games between 2010 and 2016 where a goalie played more than 30 minutes at 5v5. (This removed the games where a goalie got pulled, because we already know that in those games low save percentage caused low shot counts.) I then split the sample into even and odd halves. This is the save percentage distribution for the odd half.
Certain of the possible values are only very rarely hit, mostly the ones on the extremes – in other words, there are much fewer games with 6 to 8 goals against, less than 15 shots against, or more than 40 shots against than in our model. In fact the vast majority of results occur within an even more limited range. This affects how the statistical correlation comes out.
Most NHL games end up somewhere in the red box, which has the effect of flattening the trend line and reducing the correlation.
If all values are given equal weight, like in our model, the correlation coefficient between shot volume and save percentage is 0.38, which is pretty high as far as these things go (in other fields, this might not be a strong enough R^2 to claim correlation.) For the actual distribution, the correlation coefficient is 0.04, or basically nonexistent.
Yet, the pattern remains visible. As shots against increase, the range of observed values gets higher and narrower. The caveat is that for most games, most of the time, results occur in a very restricted range, one where the distance between points is extremely small. And as always, because we’re dealing with a range, a goalie facing 20 shots can certainly put up a save percentage higher than one facing 40 shots.
At a season level, however, things change.
Here the “possible values” for the model are approximate to observed values for 1200 to 1240 shots against. I have included this to show that the general trend of higher and narrower exists but is less dramatic than at the game level. Results outside of that range are certainly possible.
As you can see, the range of observed results gets higher and narrower as goalies face more shots in a season, but it is a much more spread out and random distribution. The relationship between more shots and higher save percentages is weaker, again in part because of the math of the situation.
The limited range of goals against expands as more games are added. At the same time the sheer number of results decreases as games are aggregated into seasons. Thus the pattern of higher and narrower, while still present, is much weaker. There is still a very slight tendency for goalies seeing higher shot volume to post higher save percentage, but the range of possible values is much bigger and the number of data points is much smaller, as you can see when we zoom in on the middle of the graph.
The correlation essentially disappears under those conditions. The tendency toward inflation is so weak at this point that there’s no way to use it to predict where a goalie’s numbers will land.
What is abundantly clear, however, is that there is no general tendency to see lower save percentage when facing more shots. The trend, even as weak as it is, is upward, although goalies have clearly posted uncommonly low numbers.
What does this all mean?
It means that shot volume is not a good explanation for a goalie’s save percentage, either in a positive or a negative way. Seeing high volumes at the game level tends to inflate save percentage a small amount, especially at the extremes, without shedding any additional light on actual performance. If your goalie is seeing a lot of shots in a specific game, it should be expected that their save percentage will be inflated.
As a goalie plays more games, even the slight tendency towards inflation weakens and disappears for all practical purposes. At the season level, the range of save percentage values is so large and the number of data points is so small that the distinction between a high shot season and a low shot season is largely unimportant for deciding what save percentage we expect of a goaltender.
In other words, looking at shot volume doesn’t get us very much closer to understanding why a goalie posted the numbers he did, but it does have an effect on how save percentage looks. It’s important information about the statistic because these mathematical realities limit how much information we can actually glean from save percentage.