Life is one decision after another.
At the small end, you have to decide between gukbap and stir-fried pork for lunch,
and at the big end, which house to buy, and whether to buy at all or take a jeonse or pay monthly rent.
You have to pick a lunch too (source: the Gimbap Cheonguk website)
Big or small, our decisions land on us.
So we want to decide well.
That’s how you get a good lunch, and a stable roof over your head.
What does it take to make a good decision?
A sound basis, for a start.
A sound basis needs good material to build on, and that material is data.
So making a smart call from data seems to need more information.
Which is to say, more information should mean better decisions.
But is that actually true?
Does having more information really help you decide?
To test it, researchers in the US ran an experiment with 8 professional horse racing analysts.
They gave the analysts 5 pieces of information about the horses (past results, times, weight, and so on) and asked them to predict the race.
Then they raised it to 10, 20, and 40 pieces and asked for the prediction again with more to go on.
“Today’s Races”, a booklet analyzing horses’ times and weights (Korea Racing Authority, Today’s Races)
Were the predictions more accurate with more information?
Was accuracy higher at 40 pieces than at 5?
Accuracy didn’t move.
At 5 pieces, at 10, at 20, at 40, it sat at about 16% every time.
More information bought no accuracy at all.
The strange part is what happened to their confidence.
The more information they had, the surer they were about their own predictions.
Accuracy didn’t move at all, but their certainty that they were right went up by more than 50%.
More information doesn’t always raise accuracy.
Sometimes all it plants is a confidence with nothing under it
A screen packed with numbers somehow makes you feel sure. (Naver Pay Securities, Ecopro)
Feeling certain because there’s a lot of data isn’t a racing analyst problem.
We do it too. The longer the report, the more references it cites, the more complicated its tables and charts, the more we believe what it says.
Even though the volume of material doesn’t raise accuracy.
Getting fooled by the sheer amount of knowledge and overrating yourself has a name, the ‘illusion of knowledge’.
Does the illusion of knowledge really matter?
What about in real life?
Even if accuracy doesn’t rise, isn’t more information still better?
In real life, more information makes the outcome worse.
Unlike an experiment, real life charges you for information.
The time and money it takes to get more information and process it can leave you worse off as the pile grows.
Ever dug through Coupang and Naver for a slightly cheaper version of something and burned more time and energy than the money you saved?
A textbook “information cost fallacy”.
Amazon CEO Jeff Bezos’s “70% rule” points at the same thing.
Bezos tells Amazon employees to decide once they have 70% of the information they want.
Waiting until you have all of it wastes time.
More information can hurt, and we still fall into the illusion of knowledge easily.
To avoid it, focus on the quality of your information, the part that actually helps accuracy.
Check whether what you’ve been handed is good enough to raise your accuracy at all.
Which is to say, the quality of your information decides your accuracy.
What does quality of information mean?
What exactly does quality of information mean? And is the extra confidence the illusion of knowledge gives you really dangerous?
Someone studied which information and how much confidence make the best decisions.
The American mathematician John Kelly.
In the next post we’ll go through the “Kelly criterion” he built and see what kind of information makes a good decision.
P.S. Quality mattering more than quantity also means half-baked information is worse than none.
Rumor-mill news, books hawking success, forum ramblings, you’re better off not reading them.
References:
-
Slovic, P. (1973). Behavioral problems of adhering to a decision policy.
-
Jeff Bezos 2016 Amazon Shareholder letter