Supernatural University: The Pitfalls of Confirmatory Bias
Supernatural University: The Pitfalls of Confirmatory Bias
I spent several hours Saturday, July 16, 2011 at the National Transportation Safety Board (NTSB) Training Center listening to a presentation on and reviewing the evidence from the NTSB investigation of the explosion of TWA flight 800 off the coast of New York, which happened 15 years ago. I can hear you asking what the investigation of a demon-free plane crash has to do with Supernatural, and the answer is this: it gave me an enhanced perspective on one of the many psychological reasons for things that have happened both inside the show in the relationship between the brothers, and outside the show in the fandom itself, including the constant Sam-girl versus Dean-girl bitchy debates over which brother is supposedly being favored or ignored on the show. Welcome to a new Supernatural University psychology seminar on the topic of confirmatory bias!
Plane Crash Investigation
Let me set the stage by sharing a bit about TWA 800. Depending on how old you are, you may or may not recall that the plane, a Boeing 747-100 with 230 people aboard, exploded in midair over the ocean shortly after taking off from JFK airport about 8:30 PM on July 17, 1996. Everyone on board died. After a painstaking investigation that included recovering the aircraft debris from the ocean floor and actually reconstructing much of the plane, the NTSB investigators determined the probable cause of the crash was an explosion in the center wing fuel tank resulting from the ignition of the flammable fuel vapor/air mixture in the nearly empty tank. The ignition source couldn't be determined with absolute certainty, but was most likely a short-circuit outside the fuel tank that allowed excessive voltage to enter the tank through electrical wiring associated with the fuel gauges. In the aftermath of the investigation, changes were made to aircraft to prevent this from happening again.
Early in the investigation however, over 500 of the 700-plus witnesses along the coast who'd reported seeing the explosion told investigators they'd seen a streak of light or a missile in the sky before the blast. Even before most of those interviews began, the FBI and CIA had joined the NTSB in the investigation because of concerns that the crash might have been a terrorist event. After all, it was only a few years after terrorists blew up Pan Am flight 103 over Lockerbie, Scotland, and after the first car bombing at the World Trade Center in New York. With the recent collapse of the Soviet Union, people were worried about Russian weapons, including missiles, being available to terrorists on the black market. The waters were further muddied by Pierre Salinger, who had previously been a U.S. Senator, the White House press secretary for both John F. Kennedy and Lyndon B. Johnson, and an ABC News correspondent, asserting a highly public claim that he had radar evidence proving a U.S. Navy vessel had accidentally shot down the plane and the government was covering it up. By the time the witnesses were being interviewed, Salinger's wild claims had made the national news, not to mention being spread like wildfire by conspiracy theorists across the budding internet.
The witnesses who said they'd seen a missile or a streak of light leading toward the plane weren't lying, but they were definitely mistaken about events. Simple physics explains most of what they saw. The plane was several miles offshore and about 14,000 feet in the air when the tank exploded. No one was gazing intently out at that random spot in the night sky; their attention was drawn to it only when they heard the explosion. As anyone knows who's done the math during a thunderstorm to calculate how far away a lightning strike is by counting off the seconds until you hear the thunder - every five seconds equaling roughly a mile - sound travels slower through air than light does. The streak of fire people saw in the sky when the sound of the explosion made them look was the trail of the burning plane itself, still climbing because the engines were still running. Had the plane been hit by a missile, there would have been two trails in the sky, one from the missile and one from the plane. However, seeing that flare moving in the sky and learning shortly afterward that a plane had exploded, the logic most witnesses leaped to was a cause-and-effect explanation based on their immediate visual perception. And for many of those witnesses, who weren't interviewed immediately but only after a number of days had passed, the news coverage with reporters speculating about the possibility of the explosion having been due to a missile just reinforced that erroneous perception.
Human Psychology v. Searching for Truth
Here we get into the background psychology aspect of things. We as human beings are hard-wired to perceive patterns; it's how we explain the world to ourselves, how we assemble our own personal narrative of events. Speaking from an evolutionary perspective, this is a huge advantage, because it allows us to make imaginative leaps to fill in the things we don't directly perceive. In seeing a pattern behind where things currently are or where we want things to wind up, we can gloss over missing data to fill in with imaginative, intuitive logic what we think would have to be there for the picture to make sense. We do it all the time. In the worlds of science, technology, and invention, that pattern-seeking sense is often the singular gift that allows us to make advances by coming up with hypothetical explanations for events and then looking for the missing pieces that would either prove or disprove our theories. In the worlds of philosophy and religion, it allows us to come up with shared, believable explanations for things we may not be able to prove empirically, such as why we exist and what our lives mean.
That pattern-seeking drive is also part of what makes questions so intrinsically frustrating to us and answers so satisfying - at least, if those answers either corroborate the pattern we believe we've seen, or provide a different picture we still can readily accept. When something happens and we don't know why it happens, we're uneasy until we can come up with a reasonable explanation, because then the world can make sense and we at least can believe that we know what to expect the next time we confront a similar situation. Our brains are hard-wired to positively need to connect causes with effects. We come up with stories that fill in the blanks - the holes in the plot, the actions we can't see - to help make sense of our lives.
However useful that patterning drive can be, however, it can also mislead us. We're so predisposed to see patterns that we often see them where they don't exist. For example, our superstitions about good and bad luck - carrying a lucky charm, playing a certain set of “lucky” numbers at the lottery, performing a pre-game ritual to improve your team's chance of winning - involve our minds having made a spurious link between unrelated events, falsely implying one will influence the other when they're actually independent. Our pattern sense also misleads us when we encounter a new event that appears similar to something we've seen and explained to ourselves before, because our immediate reaction is to accept that the two events are the same and overlook clues that they're actually different. And if we've already come up with and invested belief in an explanation - in a pattern - that seems to make sense to us, most of us are reluctant to change our existing view. We're not comfortable with the sense that the world is shifting around us, we want the certainty of already knowing the answer, and we generally don't like to admit we were wrong.
All those cautionary points factor into the concept of confirmatory bias. Confirmatory bias, or confirmation bias, is the very human tendency to search for or interpret information in a way that confirms our preconceptions. Confirmatory bias affects almost everyone, but most of us aren't even aware of it. The problem is, it gets very much in the way of finding the truth, because if the truth isn't what you already believe it to be, you're not even going to look for it.
The NTSB scientist who gave the TWA 800 presentation yesterday mentioned the effect confirmatory bias had on the investigation, and while it was a small point, it made a big impression on me. NTSB conducts its investigations in cooperation with partners, usually including companies involved in the accident because they built or operated the equipment, science labs that can help ferret out clues, and other agencies with an interest in the case. The NTSB investigators are rigorously trained to start by simply collecting facts, without forming or discussing any theories about how the accident might have happened, expressly because as soon as an investigator forms a theory of events, he or she unconsciously begins looking for the pieces of data that support that theory, and may well overlook data that would indicate something else.
One of the partner investigators, however, was the CIA's top “go to” guy for missile investigations, an expert analyst who'd spent 30 years picking apart the scientific evidence of explosions and proving which were missile strikes. Long after the other investigators weighing the preponderance of the evidence (which amounts to a LOT more than I've mentioned here), were seeing a veritable cascade of unfortunate events leading to an onboard incident triggering the explosion, the CIA investigator remained stubbornly convinced that it had to have been a missile. Eight months into the investigation, reviewing the witness statements for the bajillionth time, he finally made the connection for himself between the flash-and-sound arithmetic and the single trail of fire, and realized he'd blinded himself to the truth by his own ingrained perception, based on his history, his approach, and the myriad of statements from witnesses who insisted they'd seen a missile. He hadn't accepted the information when it came from others. It only had impact when he finally worked the equations out for himself.
Supernatural Bias
We've seen the perils and pitfalls of confirmatory bias on the part of Sam and Dean any number of times. Sometimes it's been a feature of the plot. For example, in All Dogs Go To Heaven, the guys assumed, based on the initial case information, that they were hunting a werewolf and thus stopped their surveillance at dawn, only learning later that they were up against skinwalkers, whose M.O. was slightly different. Similarly, their automatic expectations, based on their knowledge and experience, that they were dealing with certain monsters or ghosts left them surprised to learn the villains in both The Benders and Family Remains were simply warped, twisted people.
Much more ruinously, both brothers have displayed confirmatory bias in dealing with each other, based on their often mistaken beliefs and fears concerning each other. For example, in seasons one and two, Sam was always hesitant to bring up his psychic powers with Dean, not just because Sam himself was afraid of what they portended, but because he was afraid Dean would reject him as a freak. Sam often interpreted Dean's comments and actions negatively even when Dean didn't mean them that way; as just one example, look at Sam's reaction to Dean's reluctance to seek information on the psychic kids at Harvelle's Roadhouse in Simon Said. Dean was worried about what might happen to Sam if other hunters learned there was something possibly supernatural about him, but all Sam heard was Dean calling him a freak. Sam expected and feared that Dean would consider him a freak, so instead of hearing Dean's fear for him, Sam heard seeming condemnation.
Dean often did it too: remember Dean in season three - seeing Sam behaving differently and wondering, because of what Azazel had said in All Hell Breaks Loose, Part 2, whether Sam had really “come back wrong” - watching Sam with growing disquiet and even asking Bobby about his thoughts on Sam's behavior in Sin City, but not asking Sam about his behavior outright until Malleus Maleficarum (and never telling him what Azazel had said even then). Think about how much we learned from episodes as early as Skin through Shadow, Devil's Trap, and finally Dark Side Of The Moon about how Dean's perception and (mistaken) belief that he needed and cared more about Sam than Sam did about him had darkened and distorted Dean's view of events involving his brother all along the way.
In season four and much of season five, both brothers, trapped within confirmatory bias, saw only what they most feared and had thus conditioned themselves to expect from each other - betrayal, mistrust, weakness - rather than seeing the truth of each other, and acted accordingly. Their misperceptions of the truth and their fear of and inability to open up to each other and hash out a common understanding drove wedges between them that facilitated the demons' and rogue angels' plans to free Lucifer. Those barriers to and gaps in mutual understanding persisted up until their common resolve at the end of Point Of No Return, which finally started to heal their partnership and paved the way for their brotherly love and belief in each other to save the world in Swan Song.
And that brings me to the fandom aspect of confirmatory bias. I think nothing demonstrates it more than the non-stop Sam-girl/Dean-girl arguments about which character is being ignored, defamed, or favored more or less than the other. Welcome to a psychological, human understanding of what's going on. A fan who is passionate about one or the other brother and has come to a personal belief that he is being shortchanged is going to seek out and interpret every piece of available information in a way that supports that belief and reinforces that pattern, and won't even hear anything that would counter that belief. I've seen disputes from both camps using the exact same episodes and story arcs to argue each side against the middle, and both sides at once. I don't buy either side, and openly acknowledge that my brother-balance reflects my own personal bias.
Unfortunately, because this is a matter of individual belief based on personal perception rather than physical science, it isn't amenable to a purely rational, arithmetical resolution like the one that let the CIA investigator finally trump his own preconception of the cause of the plane crash. I'm afraid the differing views, with all their rancor, will still continue, since few on any side are psychologically equipped or emotionally willing to perceive the others' views.
I would hope, however, that we might all take a step back and think about the extent to which our personal bias - our human need to see understandable patterns in our existence and to find validation for the beliefs in which we've invested our hearts - may be distorting the real image of the world. And I'm not just talking about the world of the show or of fandom, but also about the world of our own real lives and real relationships.
Take the experience of the Winchester brothers to heart, and consider: what you see from your perspective as a deliberate slight may be nothing of the sort. Don't assume you know what someone will do or why they did something; always be willing to ask, rather than to accuse.
After all - what you think is a missile aimed at you might just be someone else's pain exploding in a random direction.
Class dismissed.