Earning the Vezina 2018, Part 2: Advanced stats and the rightful winner
In Part 1, we revealed who will win the Vezina award for 2018. No one can predict the working of 31 general managers’ minds with certainty, of course, but based on historic voting patterns, Pekka Rinne is very likely to win this year.
Unless a specialist goaltending magazine could make a thoroughly convincing case for a different candidate. An indisputable case. A case so powerful it somehow compelled every GM to read it, and possibly even rewrote parts of their cortexes.
What follows here in Part 2 is that very Inception.
A Rationale for Deeper Statistical Analysis
No GM can justly apply the eye test to determine the best season-long goaltending performance. In any given year, around 20-25 goalies will finish with over 50 games played, the unofficial cutoff point for Vezina consideration. Of those, maybe 10 will have very good seasons, making them potential Vezina candidates. Watching even 10 contending goaltenders through only half their starts would be a monumental time commitment it’s simply not realistic to expect (to say nothing of a person’s ability to remember and process that much information in a meaningful, just way).
So, GMs, like most fans and pundits, turn to the traditional big-four goalie statistics to help them decide: Wins. Raw save percentage. Goals-against average. Shutouts.
This is a problem.
A goaltender undoubtedly has a measure of influence over his team’s victories; the team around him, it should go without saying, has a much greater influence. The greatest goaltender of all time surrounded by a bad team would lose. A lot. A below-average goalie surrounded by a powerhouse squad would win a lot of games – does this say anything about goaltending quality? Not much.
Shutouts and goals-against average have similar issues: team effects have a massive influence, and crediting or blaming a goalie for them makes no sense. Save percentage is the one stat that the goaltender has a decent individual claim to, and fortunately, it has become the most important for Vezina voters. Unfortunately, it can also be highly deceiving.
Raw save percentage treats every shot as though it were equally difficult to stop. Saving a muffin from the red line or a perfectly-executed 2-on-0 are worth exactly the same amount. This would be fine if the total difficulty of shots faced averaged out for all goalies over the course of a season, but this doesn’t happen: goalies who play on worse defensive teams face more difficult shots overall. This means that even if two goalies sport an identical league-average 91.2 save percentage, one might have significantly outplayed the other in order to earn it.
This is why advanced statistics are important. Sites like Corsica Hockey and Hockeyviz provide important information about the context in which a given goaltender is working. Using data provided by the league about shot location, type, and situation, they show the degree of difficulty each goaltender is facing compared to his peers. Not every factor is accounted for, and team effects still play a role, but we can get a far clearer picture of a goaltender’s actual effectiveness, with significantly less influence from the team he happens to play for.
In short, advanced statistics help us to credit goaltenders for their own contributions, rather than giving them credit for their teams’. This should be the goal of any serious individual awards evaluation.
Painting a Statistical Picture
Last time, we identified Pekka Rinne, Connor Hellebuyck, and Andrei Vasilevskiy as the likely winner and finalists, respectively, of this year’s Vezina vote. Let’s compare the degree of difficulty they faced using heat maps, showing to what degree they faced more or fewer shots than league average from given locations. The more red you see directly in front of the goal and up through the slot, the tougher for the goalie. The more blue, the easier.
It’s clear that, of the three likely finalists, Rinne faced the most shots from the most dangerous location. Vasilevskiy had it easy directly in front, but had to turn aside a lot of rubber from the slot and just inside the dots. Hellebuyck, comparatively, got to float in calm blue waters, sipping a margarita and laughing at his competitors. Perhaps, for an instant, an idle thought for a Vezina rival crossed his mind like a fleeting cloud, and he wondered how John Gibson might be doing.
Dear God. It’s a bloodbath. Gibson faced more shots from the deadliest area directly in front of the goal than any of the presumed finalists, by a large margin. His heat map makes even Rinne’s look like a spa day, prompting anyone who sees this to wonder: how on earth is Gibson, who posted a save percentage just below Rinne’s, above Hellebuyck’s, and well above Vasilevskiy’s, not the frontrunner, given the huge discrepancy in shot-difficulty these maps imply? That’s a question anyone advocating for any of the likely finalists would have to answer to make a convincing argument. To my knowledge, no one has.
Beyond Shot Location, into Expectation
Heat maps are an excellent, quick, concrete indication of the difficulty of shots a goaltender has faced. In a glance, you can see massive differences between goaltenders otherwise regarded to have had similar performances. But they don’t offer the most detailed description of shot quality publicly available – that’s why we haven’t already crowned John Gibson the deserving winner and ended the show here. Corsica Hockey offers something more – an expected save percentage statistic that incorporates shot location, but also several other variables, such as whether the shot was a rebound, taken off the rush, an odd-man rush (separating breakaways, 2-on-1s), taken by a forward or defensemen, team strength (5-on-5, shorthanded) and so on.
Taking all these factors into account, which goaltender playing at least 3000 minutes (at least 50 games) faced the most difficult shots in all situations? The answer may surprise you: Of 19 goaltenders meeting the criteria, Sergei Bobrovsky topped the list, with an expected save percentage of 90.4. He beat out Cam Talbot (90.5%) and Jaroslav Halak (90.7%), both of whom (understandably) struggled this season. As for our other Vezina contenders, John Gibson had the 8th lowest expected save percentage (90.8%), while Rinne (91.4%), Hellebuyck (91.5%) and Vasilevskiy (92.0%), all finished within the top 6 highest expected save percentages, meaning they faced relatively low degrees of difficulty.
When we subtract a goaltender’s expected save percentage from their actual save percentage, we get a measure (called delta save percentage) of how much better (or worse) they did than they should have been expected to, given their individual circumstances: this helps to level the playing field, and reduce team effects. Again, this list may surprise you. John Gibson leads, outperforming his expected save percentage by 1.74%, followed by Bobrovsky (1.65%) and Rinne (1.34%). Hellebuyck (0.93%) is sixth, while Vasilevskiy is 14th, and actually just into the negatives (-0.09%, which is a bad thing).
Using their performance above expectation to calculate how many goals above average each goalie saved, we begin to see a pattern emerging. This time, Bobrovsky (having played more and faced more shots) leads, stopping 32.76 more goals than a league-average goalie would have. Gibson is just behind in second (32.51), while Rinne is a distant third (24.62). At this point, discussing the others is pointless: Gibson and Bobrovsky are battling for the lead, while Rinne is comfortably in third.
The final indispensable measure offered by Corsica is high-danger save percentage, quantifying a goaltender’s ability to stop the most difficult shots (based on the factors outlined above for expected save percentage). While medium- and low- danger save percentages see goaltenders clustered closer together (meaning there is little difference between them), high-danger save percentage shows significant distinctions between goaltenders – as such, it’s a useful indicator of quality, and facilitates meaningful comparison. By now, you’ll have ceased to be surprised by the top 3: Bobrovsky leads (83.33%), followed by Gibson (82.76%), and Rinne (82.75%).
Based on all this, I would put Bobrovsky and Gibson in a virtual tie, with Rinne in third. To further separate team effects from goaltender performance, many analysts prefer to use 5-on-5 instead of all-situations numbers. I have used all-situations numbers throughout because I believe some goaltenders are simply more skilled in penalty-kill or low-numbers situation (4-on-4, 3-on-3), and that this should be part of a complete evaluation. However, I understand the rationale, and to break the tie, I think looking at 5-on-5 numbers is justified.
In this case, it’s also definitive: at 5-on-5, Bobrovsky tops every category we discussed above: lowest expected save percentage, highest delta save percentage (by over a full percentage point, at that!), highest goals saved above average, and highest high-danger save percentage.
So, there you have it. By the best measures available to the public, Sergei Bobrovsky should be your 2018 Vezina winner. If you’re a general manager, or the person assigned by one to make this decision, do the right thing: vote Bob. The eye-test confirms it.