Today, a fun experiment and an important cognitive bias that goes with it.
(A cognitive bias means a mistake humans make often.)
The experiment is the “find the rule” study run by the American psychologist P.C. Wason.
It’s easy enough that an elementary schooler could do it.
Subjects are handed a three-number sequence: “2-4-6”.
The sequence follows some rule, something like “each number doubles” or “each number drops by 3”. The subject’s job is to find that rule.
Of course 2-4-6 alone doesn’t tell you the rule. So the subjects call out three-number sequences of their own,
and the researchers say only whether that sequence follows the rule or not.
For example,
the subject says 10-12-14 and the experimenter says “fits the rule”,
the subject says 1-4-2 and the experimenter says “doesn’t fit the rule”,
the subject says 8-10-12 and the experimenter says “fits the rule”. That’s how it goes.
Looking at those examples, did you catch what the rule is?
Which sequence would you call out to check whether your hypothesis holds?
Have you thought about it enough?
Want practical tips that make you a little smarter every week?
I cover persuasion, marketing, writing, and the other skills modern life demands.
Subscribe now and get notified when a new post goes up!
Here’s the answer.
The rule you came up with was probably “even numbers going up by 2”.
(95% of subjects thought so.)
And if you weren’t sure yet, you’d have checked whether something like 4-6-8 or 6-8-10 fits.
If you did, you got it wrong, the same as the 95%.
(I got it wrong the same way the first time I saw this!)
The actual rule is “numbers that keep getting bigger”.
Most people never find it.
Why?
Because people look at 2-4-6 and get stuck on the idea of even numbers going up by 2, so it never occurs to them to check past it. They test 4-6-8 and 10-12-14, which fit, but they never test a counterexample like 1-3-5 or 2-6-10, sequences that break their rule.
That comes from an interesting human tendency called “confirmation bias”.
Confirmation bias is the tendency to get certain easily, then go looking for evidence that backs up (that is, reinforces) the certainty.
Source: Meta Comedy Club EP.25, Lee Je-gyu, the confirmation bias guy
Confirmation bias is why people fixate on “reasons they’re right” instead of whether they’re “actually right”.
The asymmetry of evidence
The other interesting thing about this experiment is that it shows piling up confirming evidence doesn’t raise your odds of being right. You can manufacture as much evidence for a wrong hypothesis as you want, the way 6-8-10 does.
Collect 100 of them, collect 1,000, and there’s still no guarantee you’re right. In this experiment you can produce infinite evidence that fits “even numbers going up by 2”, and you’re still stuck on the wrong answer.
On the other side, one counterexample like 2-6-10 is enough to show you’re wrong.
A single piece of evidence that you’re wrong beats 1,000 that you’re right.
That’s what the asymmetry of evidence means.
In reasoning, something matters more than how many reasons you have. It isn’t an election, where the side with the most reasons wins. You have to weigh what each reason is worth, and supporting reasons don’t prove a hypothesis.
With a math problem, finding the answer by any method at all is enough. Reasoning problems differ here.
There’s no end to the reasons you can find for a stock you bought to go up. But have you thought through the reasons it might fall?
So if you actually want accurate predictions, you have to focus on the ways your hypothesis could be wrong.
Stop clinging to “even numbers going up by 2”. Look for alternative hypotheses, “odd numbers going up by 2” or “three even numbers”, and test them one at a time. If you disprove the alternatives along the way, your odds of being right really do rise and you’re entitled to more confidence. If you find a more plausible alternative, you have to be ready to overturn your own reasoning. Most people aren’t ready to give up the reasoning they’ve grown fond of, so they can’t judge objectively.
Watching the evidence for your hypothesis stack up may feel good, but your accuracy might not be moving at all.
Beating confirmation bias
Confirmation bias is hard to beat because, unfortunately, humans are built this way to begin with.
As I said in an earlier post, “Why Ray Dalio’s Principles is hard”, we treat the world we want (I’m right) and the world as it is (I might be wrong) as the same thing.
Confusing guesses, reality, and desire is human instinct, and getting past it takes staying conscious of it.
On beating confirmation bias, Charlie Munger said this.
Never argue against the other side’s case
until you can state it better than they can.
For accepting the asymmetry of evidence and beating confirmation bias, there’s no better line.
Want practical tips that make you a little smarter every week?
I cover persuasion, marketing, writing, and the other skills modern life demands.
Subscribe now and get notified when a new post goes up!
You know the saying “know the enemy and know yourself, and you will never be defeated in a hundred battles”? That version is wrong. The original line in The Art of War reads “know the enemy and know yourself, and in a hundred battles you will never be in danger”. Gathering plenty of reasons won’t guarantee your guess is correct, but paying attention to how you might be wrong will at least keep you out of danger, meaning the black swan.
One fitting counterexample is worth more than 1,000 reasons you could pick up easily. So pay attention to how you might be wrong. Keep your eyes on that and you get the hundred battles without danger.
Books referenced: Superforecasting, Clear Thinking
You might like these too!