Cathy O’Neil has a great book on big data, Weapons of Math Destruction, but one with a fundamental flaw. The flawed claim is made in the book’s subtitle and permeates throughout the book. The subtitle is: “How Big Data Increases Inequality and Threatens Democracy”. I could find no significant evidence of big data increasing inequality in the book. She shows…
Category: Big Data
Is Math Racist?
Cathy O’Neil’s book – Weapons of Math Destruction – is an entertaining and informative read. She has done a great job of highlighting the challenges with math models. (I have one massive problem with the book but I’ll detail that next time, see here). First I’ll address the question on everyone’s mind: Is Math Racist? Math Models…
The Flat Maximum And Data Science
Steven Finlay has a useful book on Data Science, (Predictive Analytics, Data Mining and Big Data). He has lots of helpful practical advice in an easy to access form. He highlights the idea of the flat maximum. The Flat Maximum This is a general recommendation to read the book. I will also highlight a point Finlay makes….
Data Without Small T Theory
Standard Deviations Gary Smith’s impressive Standard Deviations book concerns an important point. Statistically inclined people often seem to miss that theory small t matters. Smith is keen to note that data isn’t enough on its own. “Data without theory… is treacherous” (Smith, 2014, page 233). Smith describes a case of a cholera outbreak. This outbreak…
Simpson’s Paradox: Data can be very confusing
One of the strangest things in statistics is Simpson’s paradox. The paradox happens when two sets of data each show the same result. Yet, when you combine the data into a single data set the combined table gives you a different result. Data Can Be Confusing Smith explains this using a click data example. In the data, he shows when you look at…
The Three Stages of Business Analytics
Thomas Davenport is one of the best know voices in the field of business analytics. He has a book with Jinho Kim which discusses how individual business people can best manage their work in a world where analytics are a key part of many business strategies. The aim of the book is to enable managers,…
How Sexy is Working With Big Data?
I think that academics should share their opinions widely. Some academics may believe that they have no opinions, they just relate what the data says. This is might be true for extremely empirical scholars, those who typically see themselves working with big data. Such scholars are kidding themselves. We must be willing to change with data but our experience helps…
Predictive Analytics And Vast Search
Eric Siegel has an excellent book on predictive analytics and vast search. As his title suggests these involve lying, buying and dying as well as a few things that don’t rhyme. Applying Analytics The center of his book is a table of applications of predictive analytics. The marketing examples (Table 2) give a number of interesting…
Spurious Correlations: A Big Problem With Big Data?
Tyler Vigen has done great work popularizing Spurious Correlations. He has found an effective way to convey an important message. Namely, that correlation does not equal causation. Lots of things are correlated but that doesn’t mean that they have anything to do with each other. Data Dredging To create his graphs Vigen indulges in: Data Dredging… a technique used…
Uncovering The Message In The Mess Of Big Data
This week I’m focusing on research that I’ve co-authored with Xin Wang in Business Horizons. We called this ‘Uncovering the Message in the Mess of Big Data’. Our article aims to explain to managers how they can work out what the messages are in large amounts of data. What Data Should You Look At? The classic application…
