This week I’m focusing on research that I’ve co-authored with Xin Wang in Business Horizons. We called this ‘Uncovering the Message in the Mess of Big Data’. Our article aims to explain to managers how they can work out what the messages are in large amounts of data.
What Data Should You Look At?
The classic application of text mining is to analyze product reviews on Amazon or similar websites. The structured data — 5 stars or 2 stars — has value at the headline level. Sadly, knowing a product is 3.8/5 doesn’t help the manager know what to do to improve the product. The manager really wants to dig into why consumer likes about. Similarly the manager wants to know what the consumers doesn’t like about the product. How the consumer is thinking about the product more generally is also useful. To do this we analyze the text that goes with the ratings.
What Technique Can You Use When Uncovering The Message In The Mess Of Big Data?
The technique we recommend to do this is fairly well know in computer science/statistics. This means managers have a decent chance of being able to hire talent (probably Masters level graduates) to implement the technique. “Latent Dirichlet Allocation (LDA) can analyze huge amounts of text. It can describe the content as focusing on unseen attributes in a specific weighting. For example, a review of a graphic novel might be analyzed to focus 70% on the storyline and 30% on the graphics.” (Bendle and Wang, 2016, page 115).
The idea is that we can understand what consumers’ care about from what they talk about. If you modify LDA it can even assign a valance to topics. Valence measures the positive/negative terms used. You can use this to infer which topics are positively or negatively seen by consumers. Valence is important where attributes that get mentioned a lot may be thought of as either good or bad. If consumers talk about laptop speed are they complaining? Instead of compalining maybe the consumers are impressed? This difference matters.
Understanding The Market
Interestingly business strategists can use the same technique to better understand the market. As we saw the “.. value of this technique extends well beyond the CMO’s office as LDA can map the relative strategic positions of competitors where they matter most: in the minds of consumers.” (Bendle and Wang, 2016, page 115). We are living in a big data world. The good news is that there are techniques, like LDA, to help us understand the messages.
As a special bonus we posted slides on the Business Horizons website. These slides summarized the work. You could have got to hear my talents as a voiceover artist. Unfortunately I can’t find them here with the paper. I’ve posted the slides here (without audio) in case they are helpful.
Hope you enjoy it.
Read: Neil Bendle and Xin Wang (2016) Uncovering the message from the mess of big data, Business Horizons, Volume 59, Issue 1, January–February 2016, Pages 115–124