Key Takeaways
Before trusting a statistic, ask what messy reality its definition excludes
Counting starts with a choice. A statistic can look precise only after someone decides what belongs in the count. A survey that labeled one in four teenage boys serious or prolific offenders included pushing or grabbing siblings, sometimes just six times in a year. More than half of reported assaults were pushing or grabbing, and over a third involved siblings. The headline made a broad range of behaviour sound like a population of hardened criminals.
The raw count can wobble. The US Census recorded about 106,000 centenarians in 1970, while the Census Bureau estimated the true number was below 5,000. Forms, definitions, memory, and human error all shaped the result. Before accepting a count, ask what qualifies, who decided, and whether the items grouped together are genuinely alike.
What's striking is that measurement is not merely a technical step after reality has been observed. Defining a category helps create the object the statistic appears to describe. This is familiar in fields such as law and psychology, where a measurable proxy can quietly replace a richer idea like harm or wellbeing. Clear definitions make comparison possible, but rigid definitions can erase distinctions that matter. The practical test is not whether a category is perfectly pure, which is rarely possible, but whether its boundaries suit the claim being made.
Translate national totals into a share a person can judge
Zeros need a scale. A total such as £300 million sounds enormous until it is tied to what it must provide. The UK government announced that amount over five years to create one million childcare places. Divided across the places and years, it came to about £60 per place annually, or roughly £1.15 per week. That is not a sum that could pay for childcare, so the word create must mean something other than fully fund.
Make the number human-sized. The same check helped put a projected £800 million NHS deficit in context: about one per cent of the health budget, or around £16 per person per year. Neither calculation settles whether spending is wise. They reveal whether a dramatic total is large relative to the people, budget, or task it concerns.
Per-person translation is a form of dimensional analysis, a basic scientific habit of checking whether quantities make sense in relation to one another. It also exposes a political asymmetry: governments announce totals, while citizens experience costs and benefits locally. Dividing a sum evenly is not always the right model, however. A programme may target particular groups, and a national average can conceal who pays or gains. The calculation should therefore begin by identifying the relevant population, rather than assuming every person receives an equal slice.
Expect random events to form patterns before naming a culprit
Chance makes clusters. Near Wishaw in England, residents worried when nine cancer cases appeared among roughly twenty households close to a mobile-phone mast. The concentration felt like evidence of a shared cause. But scatter a bag of rice across a floor and the grains bunch in some places and leave gaps in others. Randomness does not produce an even spread, so a cluster can occur without a single local cause.
Look for the phantom target. The Texas Sharp-shooter Fallacy describes drawing a bull's-eye around a cluster after the shots have landed. If many locations and diseases are examined, some striking patterns are bound to appear by chance. That does not prove a suspected exposure is harmless. It means the cluster is a reason to investigate, not by itself a verdict about causation.
Human perception is tuned to notice patterns because missing a real threat can be costly. That bias is useful in a forest, but can turn coincidence into a compelling story when many possible patterns are available. The more places, outcomes, and time windows examined, the easier it becomes to find a seemingly remarkable cluster after the fact. A sound investigation therefore specifies what pattern would count as evidence, compares it with an appropriate baseline, and looks for independent confirmation. Scepticism protects inquiry when it remains open to evidence on either side.
Separate a real trend from a lucky swing before claiming success
Numbers move on their own. Regression to the mean is the tendency for an unusually high or low result to be followed by one closer to its typical level. The authors use a man and a dog on a long lead: the dog darts around, while its handler may continue steadily in one direction. A single change in the dog's path cannot reveal the man's course.
Policy can claim the dog's movement. Speed cameras were initially credited with a 35 per cent fall in deaths and serious injuries at monitored sites. A later government analysis attributed much of the fall to regression to the mean and a broader decline in accidents, leaving roughly one fifth of the apparent improvement attributable to cameras. The estimate was disputed, but the original claim had counted a fall without adequately separating policy effects from ordinary fluctuation.
This is the logic behind a controlled experiment: compare outcomes with what would probably have happened without the intervention. Randomised trials are especially useful because random assignment helps balance other influences between groups. Public policy often cannot wait for perfect experiments, but urgency does not turn a before-and-after comparison into causal proof. Repeated measurements, comparison groups, and transparent uncertainty make a claim more credible. The balance is important: regression to the mean can deflate an exaggerated success story, but it should not be used as a reflexive reason to dismiss a genuine effect.
An average can put most people below it
Averages blend unlike lives. The mean is the total divided by the number of observations; the median is the midpoint, with half the cases above and half below. Neither is automatically the typical experience. In 2008, President George W. Bush said the average taxpayer would face a tax increase of about $1,800 if tax cuts expired. Yet about 80 per cent of taxpayers would pay less than that. Very high earners pulled the mean upward, including the top 0.1 per cent, whose average increase was about $323,621.
Inspect the distribution. Stephen Jay Gould was told that the median survival time for his cancer was eight months. That meant half died sooner and half lived longer, not that eight months was a deadline. He lived another twenty years. The spread and shape behind a summary can matter more than the summary itself.
Averages are useful compression, but compression has a cost: the tails, clusters, and unevenness disappear. In public debate, this can convert a descriptive measure into an implied promise about what most people experience. Median values resist distortion from a small number of extreme cases, while the mean remains useful when totals matter, such as total income. Neither measure is universally superior. The right choice depends on the question, and a responsible account often needs both a summary and enough of the distribution to show who is hidden by it.
Targets can improve the score while damaging the service
A proxy is not the whole job. A target selects something measurable to stand in for a broader goal. Hospital waiting time, for example, can signal access to care, but it does not establish whether the treatment is appropriate or good. In one reported case, a patient near the four-hour emergency-department limit was admitted largely to stop the clock, while another patient in greater need had already passed the target.
Incentives change behaviour. Hospitals also reset waiting-time clocks when patients could not attend appointments, sometimes searching harder for chances to do so once targets mattered. Other systems saw ambulance calls reclassified or surgeons avoid difficult cases that might worsen survival statistics. These behaviours are called gaming: adapting actions or records to improve the measured result without achieving the underlying purpose.
This is a public-sector version of a broader incentive problem: once a measure controls rewards or sanctions, people rationally pay attention to the measure. Goodhart's law is often used to describe how a useful indicator can become less informative when it becomes a target. The answer is not to abandon measurement. Targets can focus attention on neglected problems and make institutions answerable. But they need counter-measures, independent audits, and room for professional judgement, especially when quality is multidimensional and a narrow score can shift harm elsewhere.
Ask for the starting risk, not just its percentage increase
Relative change hides the baseline. A report said that an extra ounce of bacon each day raised colorectal-cancer risk by 21 per cent. The book makes the change easier to judge by expressing it as people: roughly five men in a hundred might develop the cancer over a lifetime, compared with about six if all ate the extra bacon daily. The increase matters, but it does not mean one in five bacon eaters gets cancer.
Count positive tests, too. In an example involving 1,000 women, about eight would have breast cancer. A test with 90 per cent sensitivity and 93 per cent specificity would produce roughly seven correct positive results and about seventy false positives. Thus, only around seven of roughly seventy-seven positive results would indicate cancer. Natural frequencies, counts stated as people in a group, make base rates and test errors easier to see than percentages alone.
This is a practical application of Bayesian reasoning: the meaning of a test result depends not only on test accuracy, but also on how common the condition is beforehand. Rare conditions can generate many false alarms even with a reasonably accurate test. Presenting counts makes the arithmetic visible and can improve decisions without changing the underlying evidence. Still, absolute risk is not the only relevant value. A small increase may matter greatly across a large population, and personal choices can also depend on severity, uncertainty, and available alternatives.
Treat every sample as a mirror that can warp
Most counts are estimates. Sampling means measuring a subset and using it to estimate a larger population. During the winter of 2007, newspapers reported millions of norovirus cases by multiplying about 2,000 laboratory-confirmed cases by an estimated ratio of 1,500 community cases per confirmed case. That ratio came from an earlier study whose underlying estimate relied on just one confirmed case. Its 95 per cent confidence interval ranged from 140 to 17,000 community cases per confirmed case, an enormous uncertainty.
Ask who is missing. Migration estimates depended on interviews with a small number of passengers, mostly at major airports. As travel shifted to other airports, the survey missed many potential migrants. Sample size alone cannot fix a distorted sample. Ask how people were selected, who declined, and whether the method still reflects the population being described.
Sampling is not a second-rate substitute for a census. It is often the only feasible way to learn about a large population. The key distinction is between random error, which tends to shrink with larger samples, and systematic bias, which can persist no matter how many observations are collected. Modern survey research treats nonresponse and changing populations as central design problems, not footnotes. Even careful estimates can be valuable when uncertainty is reported honestly. A precise-looking result built on a warped sample is more dangerous than a broad estimate that admits its limits.
Check shocking outliers before letting them rewrite the story
Extreme results need context. In a climate-modelling experiment, one of about 2,000 simulations produced a possible warming of 11 degrees Celsius. Roughly half were near 3 degrees. Early coverage foregrounded the extreme result, although it was not the most likely outcome. A later report described a 4-degree rise as the experiment's most likely result. The difference is not a reason to ignore the high-end possibility; it is a reason to label it accurately.
Ask where the number sits. An outlier is an observation unusually far from the rest of a distribution. It may reveal a genuine new phenomenon, reflect an error, or represent a rare but real possibility. The crucial questions are how often comparable results occur, whether the result can be replicated, and whether the claim describes a central expectation or a remote tail.
Outliers create a tension between statistical caution and risk management. A rare event can be poorly predicted yet still deserve attention when its consequences are severe. Climate planning, for example, needs both a central estimate for routine decisions and plausible high-impact scenarios for resilience. The mistake is to present the tail as the expected outcome, or to discard it simply because it is unusual. Good communication labels probability and consequence separately, helping readers distinguish what is likely from what would be costly if it occurred.
Correlation points toward causes, but does not identify one
A shared pattern can have another source. Children with larger hands tend to read better, but age helps explain both: hands grow as children mature, and reading ability generally improves too. A correlation is a relationship between variables; it does not establish that one produces the other.
Test the tempting explanation. People with multiple sclerosis can have brain lesions, and a treatment called beta interferon reduced those lesions. But studies reported in 2005 found no average improvement in the disease's progression among patients taking the drug. The lesions appeared to be an effect or marker, rather than the cause of the worsening symptoms. Before treating an association as causal, ask what else differs between the groups and what evidence would distinguish cause from coincidence.
Causal reasoning requires a counterfactual: what would have happened to the same people, at the same time, without the suspected cause? That outcome cannot be observed directly, so researchers approximate it with comparison groups, randomisation, or carefully justified models. These methods reduce particular biases, not every possible one. The book's insistence on searching for alternative explanations is therefore essential, but correlation should not be dismissed as meaningless. It is often a valuable clue that identifies where a stronger causal test is needed.
Compare like cases, or rankings measure different things
A comparison begins with the groups. In a US political advertisement, prostate-cancer survival was presented as 82 per cent in the United States and 44 per cent in England. Yet diagnosis rates differed sharply, which can change measured survival even when treatment is not twice as effective. The comparison mixed outcomes shaped by different patterns of detection.
Selection and timing matter. To defend electronic tagging, officials compared a reoffending rate of about 4 per cent among tagged people with roughly 67 per cent among former prisoners. But tagged offenders were selected as lower risk, and their measured period was often only a few months, compared with two years for the other group. A fair test must align the people, time period, outcome, and actual alternatives being compared.
Comparability is a problem of measurement as much as arithmetic. Researchers call one aspect measurement invariance: a measure should mean roughly the same thing across the groups being compared. If diagnosis practices, eligibility rules, or reporting systems differ, a ranking may reflect those differences rather than underlying performance. Adjustments can help, but they also add assumptions that should be made visible. Comparisons remain useful when their limits are explicit, especially when simple indicators are kept separate from broad claims about which institution or country is best.
Analysis
The book's deepest claim is that numerical literacy is less about calculation than disciplined interpretation. Arithmetic can be elementary while the surrounding questions are difficult: what was counted, who entered the sample, what is the denominator, what changed besides the policy, and how much uncertainty remains? That reframing is powerful because it makes statistical scrutiny available to ordinary readers without pretending that technical expertise is unnecessary.
Its method is also a theory of civic power. Public numbers often arrive with institutional authority, compressed into a headline or a target. Asking for definitions, distributions, and baselines interrupts that authority without requiring automatic distrust. The reader is invited to replace both numerophobia and number worship with provisional judgement. This is a valuable middle position: evidence can be imperfect and still improve decisions, while confidence should track the quality of the evidence.
The book's strongest examples reveal that statistics are produced inside social systems. A survey respondent can misunderstand a question, a hospital can adjust its coding, and a department can reward the visible indicator rather than the underlying service. Data are not simply collected from reality; collection procedures and incentives help shape what becomes visible. That insight now applies well beyond government reporting, including algorithmic evaluation and workplace dashboards, where a metric can reorganize behaviour before anyone has verified that it represents the goal.
There is a limit to the book's accessible analogies. Personal intuition is an excellent alarm bell, but it cannot replace specialist analysis of sampling design, causal inference, or rare-event risk. An outcome that feels implausible may still be real, and a simple per-person division can obscure unequal effects. The strongest reading therefore treats the book's questions as a first audit, not a complete proof. Its enduring contribution is a practical ethic: use numbers, inspect their construction, and keep the uncertainty attached to the conclusion.
Review Summary
The Tiger That Isn't is praised for its accessible approach to statistics and critical thinking about numbers. Readers appreciate its clear explanations, real-world examples, and humor. The book is seen as eye-opening, teaching valuable skills for interpreting data in media and everyday life. Some reviewers found certain chapters less engaging or the writing occasionally long-winded. Overall, it's recommended for anyone looking to improve their understanding of statistics and develop a more skeptical approach to numerical claims.
People Also Read
Glossary
The tiger that isn't
A pattern created by chanceThe book's image for an apparent pattern that looks meaningful but may have arisen through chance alone. A cluster of illnesses, a run of similar events, or a striking numerical coincidence can resemble evidence of a hidden cause. The image advises readers to test the chance explanation before concluding that the apparent pattern is a real signal.
Texas Sharp-shooter Fallacy
Finding a target after the shotsA reasoning error in which someone observes a cluster in scattered data and draws a boundary around it afterward, making the cluster look like a planned target. The label evokes a poor marksman who fires repeatedly at a barn and paints a bull's-eye around the densest group of holes. It warns that patterns selected after inspecting many possibilities may be accidental.
Regression to the mean
Extreme results move toward typicalThe tendency for an unusually high or low measurement to be followed by one closer to its usual level. This does not mean that every extreme outcome must reverse, or that an intervention had no effect. It means that ordinary variation alone can produce a return toward typical values, so a change after an extreme result is not automatically evidence of a cause.
Natural frequency
Risk stated as people countedA way to express probability as counts within a clearly stated group, such as the number of people affected out of 100 or 1,000. Instead of saying only that risk rises by a percentage, a natural frequency shows how many people would typically experience the outcome. This can make baseline rates, false positives, and absolute changes easier to understand.
Confidence interval
Range expressing estimate uncertaintyA statistical range used to convey uncertainty around an estimate. In the book's examples, a 95 per cent confidence interval indicates a range constructed by a method that would include the true value in 95 per cent of repeated samples under its assumptions. A wide interval signals substantial uncertainty. It does not mean there is a 95 per cent probability that a fixed true value lies in this particular interval.
False positive
A positive result that is wrongA test result that indicates a condition or event is present when it is not. False positives matter especially when the condition is uncommon: even a fairly accurate test can produce many incorrect positive results among the much larger number of people who are healthy. The book recommends expressing test results as counts so the balance between true and false positives is visible.
Synecdoche
A part standing for the wholeA figure of speech in which a part represents a larger whole. The book applies the term to performance targets that use one measurable feature, such as waiting time, to stand for a complicated service such as healthcare. The proxy may capture something important while leaving other essential qualities unmeasured, creating a risk that the score is mistaken for the whole performance.
Enron School of Reporting
Reporting debt without wealthThe authors' label for financial reporting that highlights rising debt while omitting the wealth and income that help determine whether the debt is burdensome. The name refers to Enron's emphasis on assets while failing to disclose its debts. Applied to national or household finances, the label urges readers to examine liabilities alongside the resources and repayment capacity associated with them.
FAQ
What's "The Tiger That Isn't: Seeing Through a World of Numbers" about?
- Exploring Misuse of Numbers: The book delves into how numbers are often misused or misunderstood in media, politics, and everyday life, leading to misconceptions.
- Authors' Expertise: Written by Michael Blastland and Andrew Dilnot, it draws on their experience in broadcasting and statistics to demystify numerical data.
- Practical Guidance: It provides readers with tools to critically analyze and interpret numbers, helping them see through misleading statistics.
- Engaging Examples: The book uses real-world examples to illustrate how numbers can be manipulated or misrepresented.
Why should I read "The Tiger That Isn't"?
- Enhance Critical Thinking: It equips readers with the skills to question and understand the numbers they encounter daily.
- Avoid Being Misled: By understanding common statistical tricks, readers can avoid being deceived by misleading data.
- Broad Applicability: The insights are applicable across various fields, including politics, economics, and media.
- Entertaining and Informative: The book combines humor with education, making complex topics accessible and engaging.
What are the key takeaways of "The Tiger That Isn't"?
- Numbers Can Mislead: Numbers are often presented in ways that can mislead or confuse, rather than clarify.
- Context Matters: Understanding the context and methodology behind numbers is crucial for accurate interpretation.
- Question Everything: Readers are encouraged to question the numbers they see and seek out the underlying data.
- Practical Tools: The book provides practical tools and examples to help readers critically analyze statistical information.
What are the best quotes from "The Tiger That Isn't" and what do they mean?
- "Numbers saturate the news, politics, life." This highlights the pervasive role of numbers in shaping public perception and decision-making.
- "Chance is heartless." This quote underscores the randomness inherent in statistics, reminding readers that not all patterns have meaningful explanations.
- "Averages are like that: in trying to tell us something about an entire group, they can obscure what matters about its parts." It emphasizes the limitations of averages in representing complex realities.
- "The alignment of power and abuse is not unique to numbers, but it is just possible that it could be uniquely challenged." This suggests that while numbers can be manipulated, they also offer a unique opportunity for transparency and accountability.
How do the authors suggest we interpret averages in "The Tiger That Isn't"?
- Averages Can Mislead: Averages often mask the diversity and variability within data, leading to oversimplified conclusions.
- Consider the Distribution: It's important to look beyond the average to understand the full distribution of data points.
- Contextual Understanding: Averages should be interpreted in the context of the data set they represent, considering factors like outliers and skewness.
- Practical Examples: The book uses examples like income distribution and pregnancy duration to illustrate how averages can be misleading.
What is the "Tiger That Isn't" concept in the book?
- Illusion of Patterns: The concept refers to the human tendency to see patterns or causes where none exist, often due to random chance.
- Statistical Misinterpretation: It highlights how people can misinterpret statistical data, mistaking random variations for meaningful trends.
- Vigilance Required: Readers are encouraged to be vigilant and skeptical of apparent patterns in data, questioning their validity.
- Real-World Implications: The book provides examples, such as cancer clusters, to show how this misinterpretation can lead to false conclusions.
How does "The Tiger That Isn't" address the issue of risk perception?
- Misleading Risk Statistics: The book discusses how risk is often presented in misleading ways, such as using relative percentages without context.
- Understanding Baseline Risk: It emphasizes the importance of knowing the baseline risk to accurately assess the significance of risk changes.
- Practical Communication: The authors advocate for communicating risk in terms of natural frequencies, making it more relatable and understandable.
- Real-Life Examples: Examples like the risk of cancer from processed meat are used to illustrate how risk can be misrepresented.
What does "The Tiger That Isn't" say about the use of targets and performance indicators?
- Targets Can Be Misleading: The book argues that targets often fail to capture the full complexity of what they aim to measure.
- Gaming the System: It highlights how targets can lead to gaming, where individuals manipulate data to meet targets without genuine improvement.
- Partial Picture: Targets provide a narrow view, often missing important aspects of performance or quality.
- Need for Comprehensive Measures: The authors suggest using multiple indicators to get a more accurate picture of performance.
How does the book explain the concept of "regression to the mean"?
- Natural Fluctuations: Regression to the mean refers to the natural tendency for extreme results to move closer to the average over time.
- Misinterpretation Risk: It warns against attributing changes to interventions when they may simply be due to this statistical phenomenon.
- Practical Implications: The book uses examples like sports performance and medical outcomes to illustrate how regression to the mean can be misinterpreted.
- Critical Analysis: Readers are encouraged to critically analyze data trends, considering the possibility of regression to the mean.
What insights does "The Tiger That Isn't" offer on sampling and data collection?
- Sampling Challenges: The book discusses the challenges and potential biases in sampling, which can affect the accuracy of data.
- Representative Samples: It emphasizes the importance of using representative samples to ensure reliable conclusions.
- Real-World Examples: Examples like immigration statistics and economic growth figures are used to show how sampling can lead to errors.
- Awareness of Limitations: Readers are encouraged to be aware of the limitations and potential biases in data collection methods.
How does "The Tiger That Isn't" address the issue of data quality?
- Data Integrity: The book stresses the importance of data quality and integrity in making informed decisions.
- Human Factors: It highlights how human error and biases can affect data collection and interpretation.
- Consequences of Poor Data: Examples like the Bristol Royal Infirmary case show the serious consequences of poor data quality.
- Improving Data Practices: The authors advocate for better data practices and a culture that values accurate and reliable data.
What does "The Tiger That Isn't" say about the use of league tables and rankings?
- Comparative Challenges: The book discusses the challenges of using league tables and rankings to compare performance across different entities.
- Definitional Differences: It highlights how differences in definitions and contexts can lead to misleading comparisons.
- Complexity of Measurement: The authors argue that complex systems cannot be accurately captured by simple rankings.
- Need for Nuanced Analysis: Readers are encouraged to seek nuanced analysis and consider multiple factors when interpreting rankings.
Download PDF
Download EPUB
.epub digital book format is ideal for reading ebooks on phones, tablets, and e-readers.