Cognitive bias: survivorship bias

The latest in our series on cognitive biases covers the survivorship bias. This is an error that comes about from focusing on surviving examples (and ignoring examples that are not available).

One of the most interesting examples of this bias comes from World War Two. US aircraft would fly over enemy territory to drop bombs. Consequently, they would be shot at and the aircraft would have visible damage from enemy fire which could be inspected when they returned to base. Some would, however, not make it back if enemy fire succeeded in hitting critical parts of the plane.

A conventional response was to look at the returning planes and patch up the damaged areas with additional armour. You can see an image of hypothetical data for where returning planes tended to get struck here. Abraham Wald, a statistician, took the survivorship bias into account and noted that the planes that did not return were the ones being hit in critical areas that could not be survived and it was these areas that needed extra armour. The ones that had made it back were able to sustain such damage and therefore armour there provided no additional critical defence.

In this case the language seems apt, survivors were being focused on rather than those that had succumbed. Advice was then being built upon this bias until the bias was spotted and corrected for.

Another example that is worth considering relates to how successful people are presented in everyday life. One of the wealthiest people in the world is Mark Zuckerberg whose life is often simplified down to the phrase “college dropout who founded Facebook”. The staggering levels of success achieved have often led to questions about the value of higher education and whether people should pursue entrepreneurial ambitions instead. To really consider this in greater detail, we would need to know more about people who left college to set up a business and had less success (or even failed outright). If we wanted even broader data, we would look at people who completed their college education and set up businesses.

It is therefore important to consider whether decisions are being made with due consideration of the data sample being used. As we’ve seen, sometimes data can be ‘cherry-picked’ (Mark Zuckerberg) or used simply because it is the only available data (World War Two US bomber aircraft data). Being aware of this bias means we can consider whether we can get data from a broader range than initially available. In effect, we should try to ensure that we are seeing the whole picture.

How can this affect everyday education-related situations? Consider the example of a student in your child’s class who seems to be performing at a higher level than everybody else. You decide to speak to her parents to see if they can share the secret to success, which, it turns out, they are happy to do. Before you implement the tip provided, it may be worth considering whether anybody else has tried using the same technique with different results. It’s unlikely that academic success can be simplified to single technique but hopefully this helps to illustrate how ordinary situations such as this can fall prey to survivorship bias.

Next time, we shall take a look at confirmation bias.