Base Rate Neglect and Why Do we Fall for it?

I recently finished reading "Thinking, Fast and Slow" by Daniel Kahneman. It is one of the best books I have read in my life, and I wholeheartedly recommend it to everyone. Daniel Kahneman talks in a riveting manner about various cognitive biases and fallacies that influence our thinking. These fallacies and biases hinder us from making rational and correct decisions. The book is full of interesting examples and case studies. One fallacy particularly appealed to me. It is known as base rate neglect. 

What is Base Rate Neglect?

Let me explain it using a simple example. This example is similar to one given by Daniel Kahneman in the book. Consider a person X. She is an introvert who likes to read and is interested in history. She is kind, helpful and organized. She is also the president of a local book club.

What is more likely – X is a doctor or a librarian?

If you answered librarian, then you committed a fallacy known as base rate neglect. Librarian is the wrong answer. You might be thinking why because the characteristics of X are more common in a librarian than a doctor. Let me show why Librarian is the wrong answer.

I think it is safe to say that you live either in a developed country like the USA or Germany or a developing one like India or Bangladesh. Your country has several cities, towns, and villages. Most of the cities or towns have one or more libraries, and each library has at least one librarian. There are also numerous small and big hospitals in every city, and each hospital has several doctors. Now if I were to ask you to guess the ratio of doctors to librarians in your country, chances are you will guess a number between 5 and 20. It doesn't matter where you live because it is safe to say that there are 5 to 20 doctors for every librarian in your country.

Now lets come back to our question (guessing the profession of X). In the question, there is no explicit information about the profession of X. All we have is some characteristics of X. We also have a reasonable estimate of the number of doctors and librarians in our country. But we also have certain stereotypes of doctors and librarians. X fits the bill of a typical librarian more than that of a typical doctor. This leads us to think of X as a librarian even though there are 5 to 20 times more doctors than librarians in our country. In the absence of any concrete information about her profession, the chances of her being doctor are 5 to 20 times than that of being a librarian. This is an example of Base Rate Neglect. When faced with a description that matches a stereotype, we ignore the probability and go by the description.

Let us consider another example. X studied sociology at the university. She cares about women rights. She participated in many protests regarding equal pay for men and women when she was a student. What is more likely?

Option A – "X works at a bank" or

Option B – "X works at a bank and is the president of the women wing of the bank labour union"

If you think the second option is more likely, then the mighty base rate neglect has again defeated you. "X works at a bank" is a superset of "X works at a bank and is the president of women wing of the bank labour union". Option A encompasses option B but not the vice-versa. All the presidents of the bank labour unions are bank employees but the reverse is not true. So the right answer is option A because it is more likely.

Daniel Kahneman conducted several experiments among his students. His results were surprising. In most of his experiments, less than 30% of the respondents accounted for base rates in their answers. I was surprised because I thought the numbers were remarkably low. One reason I could think of this high neglect was the time of the experiments. Most of these experiments were done in the 1980s and 1990s when cognitive biases were not so well studied. I thought fewer people would fall for the base rate neglect now because it is more commonly talked in mainstream media. I was sure that the awareness has increased from the time Daniel Kahneman did his experiments because cognitive biases and fallacies have become popular topics.

So I did what any researcher does to test a theory. I did an experiment. I conducted an online poll and asked people their opinion. I was so sure that people won't fall for base rate neglect that I purposely made the poll difficult. I asked one question with 4 possible options instead of 2 that Daniel Kahneman used in his experiments.

The poll was on a hypothetical cricket player (batsman) who had a strong resemblance to a Pakistani batsman of the 1990s and 2000s. I put a caveat that the poll is for people who understand cricket. I intentionally did it to pique the curiosity of the audience. One of my good friends (Lakshaya Chawla) saw through my ruse and complimented me on using it :). I posted the poll on LinkedIn, twitter, facebook, and few WhatsApp groups. A total of 99 people responded to the poll, and I am grateful to each of them for answering the question. Thank you!

Poll Results – Is the Base Rate Neglect Still Strong?

The poll results were totally unexpected. Less than 20% of the respondents answered correctly. Everyone else neglected the base rate. In the above question, option B is a superset of all the other 3 options and hence the correct answer. Around 70% of the respondents chose option A which is similar to the real-life story of the Pakistani batsman. After reading the question, 70% of the respondents ignored the base rate and decided to go with option A because it supported the opinion they had made about the batsman after reading the question. This poll shattered my belief that people have become more aware of the cognitive biases over the years and showed that the base rate neglect is still running strong.

Why do People Fall for the Base Rates Neglect?

A cognitive error known as "Story Bias" is one of the main causes of the base rate neglect. We (human beings) like stories. Stories move us, while cold statistical facts leave us unaffected. We love to create stories behind every phenomenon. We are influenced by narratives. And all of us stereotype, either consciously or unconsciously, to a certain degree.

Another cognitive error called "Framing" is also responsible for the base rate neglect. Our answers depend on how the question is framed. Sometimes a subset can seem larger than the entire set. I specifically framed the question in the poll to make the subsets larger than the entire set. Unfortunately, many people fell for it.

Another error known as "Confirmation Bias" is also at play here. We actively look for the evidence that confirms our beliefs. We fail to see or chose to ignore the evidence that rejects our beliefs. As a result, we look for data to support our theories rather than forming theories after examining the data. It may have happened that some people already formed an opinion about the identity of the batsman after reading the question in the poll. Then after seeing all the options, they chose Option A because it corroborated their opinion.

I find base rate neglect fascinating because it refuses to capitulate. Even those with training in statistics commit base rate neglect. Although it is difficult to always avoid it, the knowledge of base rate neglect has the potential make our lives easier. This knowledge makes us aware of our biases. And when we are aware of our biases, we think twice before making important real-life decisions, and that leads to fewer mistakes. That is one big advantage of knowing about cognitive errors because we fall prey to it less often (hopefully).