I enjoyed the book Super Crunchers. It is a couple of years old [in 2014] but it still reads well. Perhaps some of the surprise people might have had when it was published may have dissipated. In recent years things only the rare few knew about, e.g. metadata and data warehouses, have become staples of dinnertime conversation. It is useful to highlight thinking by the numbers.
Academics Are Picky
When reading the book occasionally I found myself being a bit too academic. Is thinking by the numbers really that new? Sure we have more data available to crunch now but smart people have used numbers to influence their thinking for many years.
I also worry that occasionally Ian Ayres’ tone is a bit too positive. I would like to see a bit more on the downside of applying prediction models based upon data without understanding causation. Experts, like Ayres, know the flaws in their approaches. There are, of course, flaws in all approaches. Yet, when they write about using data they sometimes downplay the flaws to try and convince the skeptical. This is a problem.
Non-experts on seeing the pleas to use data may either 1) continue to reject the plea anyhow, or 2) accept the idea wholeheartedly. The later position ignores the fact that prediction models have to make significant assumptions. For example, a typical assumption is the future will reflect the past. This is often reasonable but when the assumption isn’t correct the result can turn out to be spectacularly wrong. Prediction models generally give something to think about and learn from, most have something useful to say but none are “the truth”.
Thinking By The Numbers
That said as Ayres details there is much to be gained from understanding the insights that data allow. This is especially true when data analysis is combined with testing. He tells the story of Don Berwick who came up with 6 simple medical reforms. He then got 3,000 hospitals to test them. The use of multiple hospitals is critical.
It is hard to assess at a single hospital with 10,000 admits whether any mortality decrease is just plain luck. Yet when the before-and-after results of 3,000 hospitals are crunched, it’s possible to come to a much more accurate assessment of the aggregate impact…. In just eighteen months, the six reforms prevented and estimated 122,343 hospital deaths.
Ayres, 2007, page 86
How many more lives can be saved by showing the effectiveness of the approach?
With such a dramatic impact that can be assessed using data analysis, it is easy to agree with Ayres. Thinking by numbers is the (somewhat) new way to be smart.
For more on analytics see here, here, and here
Read: Ian Ayres (2007) Supercrunchers: Why Thinking By the Numbers Is The New Way to Be Smart, Bantam Books