Imagine you’re in charge of the Allied air force in World War II. Hundreds of thousands of lives ride on your decisions. Your job is to reinforce the bomber design so it can take German fire. Armoring the whole plane would be safe, but it would get too heavy, so you can only reinforce one or two of the weakest spots.
Luckily you have something to work from. The red dots in the picture below mark where bombers that flew in combat took hits. They cluster in the middle of the fuselage and at both wingtips.
https://en.wikipedia.org/wiki/Survivorship\_bias
So where do you add the armor? You probably picked the middle, where the dots pile up.
That’s where the most bombers got hit, so it seems like the obvious answer. But is it?
The trap in this problem is that the planes that flew out and never came back are missing from the data.
Planes shot in the wings or the fuselage made it home. Planes hit in the engine or the cockpit didn’t, so they never entered the count at all.
That’s survivorship bias. You look only at the information in front of you and never even wonder what’s missing.
The dead don’t talk, so you only hear the survivors. Stop listening for the dead and you can miss the most important part of the truth.
Want practical tips that make you a little smarter every week?
I cover persuasion, marketing, writing, and the other skills modern life demands.
Subscribe now and get notified when a new post goes up!
Is a wingsuit safe if nobody gets injured?
Wingsuits are a similar example.
You put on a suit that makes you look like a flying squirrel and jump out of the sky. As you can see, it’s very dangerous.
There’s also a joke that wingsuits produce far fewer injuries than other sports. Why? Because an accident during a wingsuit flight usually kills you on the spot, so there is nobody left to be injured. Pros with hundreds of flights behind them have died instantly. So a low injury count is no reason to call the sport safe. It’s the bomber problem again. It also runs into the unknown unknowns from the last post. To see the whole picture, you have to see what isn’t in front of your eyes.
Survivorship bias, or the habit of judging by what you can see, is common around us. You look at people who made money on crypto futures and start thinking about an extreme bet of your own, because the people who lost everything on crypto futures are invisible. They don’t get talked about online, and a lot of them genuinely disappear. A coin flip, long or short, always pays somebody, so with enough people repeating it, some lucky person ends up rich. That’s probably not skill. Same logic as the lottery.
I’ll cover how to separate luck from skill another time, but the best way to confirm skill is to see whether it repeats over a long stretch. After that, look at the thinking behind the actions. (Howard Marks covers this in The Most Important Thing.)
Both checks come down to seeing what isn’t in front of you, and doing that means constantly asking what you’re missing. As I said in the last post, you have to imagine the unknown unknowns you have no idea about and stay careful.
Look at what you can’t see
We tend to draw our conclusions from whatever information sits in front of us. Survivorship bias cuts out the data you can’t see and walks you into a bad call. Judge from the bombers that came back, or from the entertainers who made it, and you get it wrong.
To get past survivorship bias, pay attention to the data you can’t see. Before you decide, stop for a moment and ask what information you might have missed, and what information would change your mind. The world isn’t that simple, so dig for the deeper reason, watch for the point where your thinking stops, and keep the chance that you’re wrong in view.
Want practical tips that make you a little smarter every week?
I cover persuasion, marketing, writing, and the other skills modern life demands.
Subscribe now and get notified when a new post goes up!
References: The Most Important Thing, Fooled by Randomness, Wikipedia
You might like these too!