When can you trust statistics?
When can you trust statistics?
Made in partnership with The Open University, with OU consultant Dr Karen Vines, Senior lecturer in Statistics, The Open University. TIM HARFORD: We're surrounded by statistical claims, whether it's a striking graph going viral on social media, a politician claiming that life has gotten 62% better while they've been in office, or just something everyday, like a piece of economic data, the numbers are everywhere. CAPTION: HOW TO MAKE STATISTICS ADD UP MADE IN PARTNERSHIP WITH THE OPEN UNIVERSITY TIM HARFORD: So what should we think about them? Here's one approach: don't trust any of them. Mutter something about lies, damned lies and statistics. Or joke that 98% of statistics are made up on the spot. That seems smart. After all, nobody likes to be taken for a fool. But we can't just reject all statistics equally. Statistics can show us things about the world that we can't perceive in any other way. Instead, we have to exert a bit of brain power to figure out what is true and what is not. That sounds hard, but perhaps it's not as hard as you might think. So here are three easy rules to make statistics add up the three Cs of data wisdom. First, be calm. Most statistics come packaged up with emotional baggage. They're supposed to make us angry or joyful or afraid. Staggering deficits, shocking rates of crime, inspiring sums raised for good causes. These emotions are why numbers go viral on social media, the reason they end up in the headlines. There's nothing wrong with feeling emotions, but they don't help us think clearly. So before you share a claim that has you hot under the collar, take a moment to notice your instinctive reaction from rage, to denial, to vindication. Once you've noticed it, look at the statistic again. It may seem different now. Second, get context. For example, when the UK Health Secretary said in 2020 that the NHS would save £100 million over five years if everyone who was overweight lost a few pounds. What should we make of that claim? What does he mean by overweight, for example? And what evidence does he have to support this claim? But the most important piece of context is simply to understand whether £100 million is a big number. It sounds big. But there are 67 million people in the UK, so £100 million is just £1.50 each. You can figure that out on your phone. Now remember that £1.50 each was spread across five years. So that's 30 pence per person, per year. The Health Secretary then said that if every overweight person lost some weight, the NHS would save the equivalent of 30 pence per UK resident per year. Not a lot. Statistics can be a very complex subject, but you can get a long way with simple questions about context. What's being measured here? Is it going up or going down? Is it big or small? What's the source of the claim? You don't need a lot of fancy maths, just a search engine, the back of an envelope, and a curious mind. Which brings me to the third important principle - be curious. When we use a number as a weapon in an argument, or a prop for our preconceptions, we learn nothing. Instead we should think about statistics as a tool to understand the world, like a telescope for an astronomer. Ask yourself what a statistical claim is really telling you about the world and what questions it inspires. Of course we don't have time to do all this for every claim we see, so a final habit is to ask yourself whether the source of a statistical claim is respecting the three Cs. A good journalist will help you be calm, will give you context, and will feed your curiosity. A viral 'gotcha' circulating on Twitter or WhatsApp will often do the opposite. We shouldn't just accept statistics unthinkingly, but neither should we dismiss them without thinking either. Three simple habits - be calm, get context, and be curious can help the world add up.
