Putting the Eye Test to... well... the Eye Test

The "eye test." Some know it more as the best method for evaluating players, and some think it's worthless. How good are we at judging talent without the aid of numbers? The answer is complicated.

Before we get started, I have to give credit- the idea and inspiration for this article largely goes to Malcolm Gladwell, a writer for The New Yorker, and I'm going to rely heavily on his research. If this subject interests you, I couldn't recommend his book Blink highly enough. Or any of his books, for that matter.

With that said: let's surge forward.

Judgement is a very odd and fickle creature. It's subject to a number of different phenomena, some of which have names, others of which don't.

Negatives

Let's start with a few problems with "The Eye Test"

Bias

Was Parise's play dirty? Not particularly, but fans of the Ducks saw it that way and remember it that way because of their bias. Same story for Kesler's hit: was it dirty? Not as much as Wild fans remember, but we all think so because A) it was unnecessary, B) We're protective of FBJ, and C) Ryan Kesler is a dingleberry who clearly takes cheap shots and more skilled players (see my bias there?)

Bias is less a problem with judging something in the moment, and more a problem with recalling events. Our biases color our memories and can completely alter the events we recall happening. I mean that literally: there are documented cases of two subjects recounting completely different tales, largely due to their bias.

Priming

Priming is related to Bias, but is slightly different. While Bias deals with recall, Priming can change what we see, hear, and feel in the moment of the experience.

Priming is phenomenon in which your brain changes what it sees or hears based on what has happened right before it. A common example of this can be found in pop culture. People listen to songs backwards and hear messages hidden within music (which is called "back masking").

With the exception of the Pink Floyd example (which is a true case of back masking), all of those are just sounds. But looking at the words primes your brain to hear them, and so you do.

Put another way: when you think players are "good," "bad," "dirty," etc, you are more likely to see what they do that way. This can cause errors in judgement. When that player you love throws a big hit, you tend to think "yeah!" rather than "well gosh, that puts you well out of position to stop the goal that's coming..." We're all guilty of it, and it's almost impossible to avoid.

Priming is a big problem with the eye test, as it affects teams, referees, and pretty much everything and everyone in the sport. Priming is the reason a player's reputation can affect what a referee will or won't call, among many other things.

Negativity Bias

"Negativity Bias" has a few different names, but for hockey this one suits. In short, this is the tendency of humans to remember negative events more than positive events. In other words: you don't remember the good things someone does, but the bad one sticks in your head.

The player whom this most directly applies to for the Wild is Jared Spurgeon. Spurgeon, or "Tank" as we affectionately call him here, has a lot of knocks against him. The main one is his size, but there are a few others. Despite the many,

The bias towards remembering negatives more than positives might seem like common sense, but it defies a bit of logic. Humans like being happy, so why wouldn't they fixate on the positive? Research indicates that it may be evolutional at its core. Interestingly, the needed ratio to counterbalance this effect is 5-to-1; five positives for every negative. Even then, the observer has to recognize events as positive- so the subtleties don't count. In hockey, for most fans, that means it needs to be 5 stand-out plays for every stand-out mistake, and often that just doesn't happen.

Now that we've identified some of the problems with human recall and judgement, let's find some good things about it.

Positives

Intangibles

Don't worry, I'm not about to wax philosophic about the glory of things like "grit," "compete level," or "clutch-ness." But there are certain things that simply aren't measurable. For instance, if a player truly is splintering a team in the locker room, regardless of that player's metrics, they are bad for the team.

Another factor that can't be measured; if a player serves as a good mentor for others, he can improve the team beyond what his stats alone show. I think this is probably rarer than many think, but it is certainly possible, and likely happens without many knowing about it. This leads directly in the third and final intangible:

A note: while the effects of some intangibles can be measured; a mentored player will have better statistics, a loss of 'chemistry' will result in a drop in numbers, etc. these are measurements of the effects, rather than the trait itself.

Thin Slicing

Thin Slicing is the process by which humans extrapolate information automatically. There have been some studies regarding students' rating of professors which show this phenomenon well. One group of students rated a professor after a full semester of class. Another group of students rated him after a 15 minute video of the professor's class, another after a 10 minute video, etc. By the end, there were groups of students watching 10-second clips of the professor without any sound, and rating him. The results were surprising. The students who were watching short clips without sound rated the professor very similarly to those who had sat through an entire class.

In another case, a man brought a statue to the Paul Getty Museum in California. He claimed the statue was the oldest of its kind, and a number of experts confirmed its authenticity after months of research. A few other experts stated with absolute certainty that the statue was a forgery without have examined it at all, beyond simply seeing it from a short distance away. Upon further research, small issues appeared with the statue. It was made out of an incorrect material, techniques weren't correct... suffice to say eventually many experts became convinced it was fake.

Malcolm Gladwell runs through a number of situations where thin slicing comes into play- but the results are all the same. In many cases, decisions made with little or no information were as good as or better than decisions made with lots of information. It's important to note: Gladwell notes than often (though not always) the snap decisions were made by experts, with many thousands of hours of experience.

For us hockey fans, this gives some legitimacy to the "eye test." Experts could conceivably make great decisions without paying attention to in-depth analysis.

Unfortunately, thin-slicing is subject to a number of problems, including priming, biases, and some others.

A Delicate Balance

Ultimately, proper and correct analysis comes out of a balance of judgements based on experience, and cold, hard, data. This isn't the "sexy" conclusion by any means, but ultimately, data has a lot to tell us about the game we love, but doesn't discount the experience of some other things going on.

A conclusion Gladwell comes to at the end of Blink is (spoiler alert): in general, if a decision is very straightforward and uncomplicated, careful analysis and deliberation probably is the best decision-making paradigm. If a decision is complex and has many things to balance a juggle, our unconscious reflexes and instincts (after being trained by experience) may be best.

Much like Corsi and Fenwick, however, understanding how decisions are made is only a tool. What's more important is knowing how to apply it, and simply being aware can sometimes improve decision making. Furthermore, understanding that instinctive judgements are sometimes very accurate still requires you acquire the requisite instincts. The challenge, then, is to improve understanding in order to improve our instincts.

Judgement is a funny thing, and can prove to be brilliant or disastrous if decisions are made incorrectly. Hopefully as time passes, we can improve our tools to effectively make decisions. Or at least, hopefully GMCF and Mike Yeo can.