Predictive Analytics and Vast Search

Eric Siegel has an excellent book on predictive analytics and its applications. As his title suggests these involve lying, buying and dying as well as a few things that don’t rhyme.

The centre of his book is a table of applications of predictive analytics. The marketing examples (Table 2) give a number of interesting applications. Analytics can predict product choice and purchase. He talks about churn prediction and how identifying customers at risk of ceasing to be customers can be a very effective use of analytics. Movie recommendations on Netflix use analytics, as do the coupons that supermarkets such as Tesco offer you. Some will find the fact that Tesco predicts what you want a bit scary. I’m personally less concerned; I’m proud of the amount of chocolate I eat and don’t mind who knows it.

He has interesting stories about how analytical teams can work together when various elements of the team bring complementary skills.

One thing I like is that he isn’t simply a booster for predictive analytics and big data more generally. He draws attention to the challenge of vast search, or the multiple comparisons problem. “The casual “mining” of data… often involves vast search, making it all too easy to dig up a false claim” (Siegel, 2016, page 140). He suggests the problem isn’t bigger data but wider data. The problem is not that you have a lot of data, it is that you have a lot of different things that you can test. If you test enough potential connections some will look statistically significantly merely by chance. Be careful.

Siegel has a great book — I am very happy to recommend it.

Read: Eric Siegel (2016) Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie or Die, (Revised and Updated) Wiley.