PROBABLY OVERTHINKING IT

A data scientist explains the common pitfalls of statistical analysis.

Sometimes we seem to be drowning in data and statistics that are complex, contradictory, and opaque. Downey, a professor emeritus of computer science at Olin College and author of several books on computer programming, sets out to make understanding it easier. In this book, the author works his way through a number of problems. He sees statistical analysis as an important and useful tool for decision-making, but he admits that things can go terribly wrong, regarding issues ranging from health diagnoses to chess rankings to earthquake predictions. He examines common errors like the base rate fallacy, selection bias, and length-biased sampling, using visualizations rather than equations wherever possible. The chapter on how flawed analysis affected the early attempts to track the spread of the Covid-19 pandemic is particularly illustrative—and worrying. Modeling can generate counterintuitive outcomes, such as Simpson’s paradox, in which aggregates do not (apparently) match the components. A key issue in statistical analysis is finding the right place in a dataset to start, and Downey does not provide much guidance on this element. He assumes that publicly available data is reliable, but this is not always the case, and politicians and advocates from across the spectrum are usually experts at manipulating numbers to support their own conclusions. Guidance from Downey on how to separate valid source data from misleading material, perhaps in a concluding summary chapter, would have been useful. Broadly, we might have expected more practical advice from an author who clearly knows his subject. This is an interesting book, but many sections require a close reading and a basic familiarity with the math, so it is not for everyone.

Downey does a solid job of explaining why statistical analysis can fail, but overall, the book is a mixed bag.