Net Promoter and lessons for academic research

I value academic work that speaks to the issues of managers and others outside of academia. The Net Promoter Score/System (NPS) is widely used by managers and so it can be valuable when academics look into this metric.

We (I worked with Charan Bagga now at Calgary, and Alina Nastasoiu now at Booking.com) reviewed the academic literature to see what we could learn from the examination of NPS by academics . Disappointingly there isn’t much and, with notable exceptions, the work that has been done isn’t great. This is probably largely a function of incentives. It is hard to get good data to test NPS. It is much easier for academics to focus on testing what they can readily test rather than what might be useful for managers.

Some supportive evidence for the use of NPS has been produced but it isn’t especially compelling and major criticisms have often been ducked by NPS’ advocates. As such my reading is that those academics who know about NPS generally aren’t excited by it. Despite this, one challenge we saw is that some academics seek to connect with managers by noting they have heard of NPS but without digging into it. This risks implicitly endorsing claims. We would argue it is better not to repeat claims if one hasn’t properly evaluated them.

The biggest concern the literature review highlighted was the problem of unconstrained dependent variable choice. In essence if you don’t decide what you are hoping to show before starting a test you are very likely to find something you can, post-hoc, decide to highlight. If you don’t mind whether you connect a metric, say NPS, to profits, revenue, sales volume, awareness etc… you are bound to find a positive relationship between the metric and something.

Such casualness about metrics is particularly frustrating when research is said to contradict a prior relationship. For example, a paper (by De Haan and colleagues) purported to contradict the earlier research of Keiningham and his team which criticized NPS. The newer paper said it found that NPS is an effective predictor of customer retention. This is infuriating as it gives the impression of academic confusion but as we note “….Keiningham and his colleagues were testing NPS against revenue growth (Keiningham, Cooil, Andreassen, et al. 2007) to investigate Reichheld’s original claims (Reichheld 2003). De Haan and his colleagues consider NPS’ impact on retention and do not explain why they think retention and revenue growth are the same metric. The results of the two studies cannot be contrasted as the DVs are different”. (Bendle, Bagga, and Nastasoiu, 2019). Tests of different relationships can’t really contradict each other.

NPS is a popular metric and more research on its uses and misuses would help academics communicate with managers. Such research would be useful but it can only be so if academics are careful about the messages they send.

Read: Neil Bendle, Charan Bagga, and Alina Nastasoiu (2019) Forging a Stronger Academic-Practitioner Partnership – The Case Of Net Promoter Score (NPS), Journal of Marketing Theory and Practice, 27 (2), https://www.tandfonline.com/doi/full/10.1080/10696679.2019.1577689