A question I think should be asked more in academic marketing research is “how do we define performance?” Lots of people do analysis that aims to show what creates good performance. Still, there is surprisingly little attention paid to what good performance means. What performance outcomes do marketers use?
What Performance Outcomes Do Marketers Use?
Constantine Katsikeas and colleagues looked at how marketing performance is measured in academic marketing papers. They divide up papers as showing a strong theoretical rationale for the choice of measure (or not). It was rather depressing.
… the vast majority of studies covered in our review (>92%) either fail to provide a clear definition and rationale for the conceptualization of performance adopted or are inconsistent across their conceptualization and operationalization of performance.
Katsikeas et al. 2016, page 8
This translates as scholars are just using what metrics they have to hand. They are not giving serious thought to what they should use. It gets even more worrying.
…researchers may simply operationalize performance on the basis of outcome variables that post hoc provide the strongest empirical results.
Katsikeas et al. 2016, page 4
Such actions pretty much invalidates the findings. Typically tests assume that there is a strong theory to support or refute. If you don’t have a strong prior theory when you find something you can’t call it a test of a theory. It is just exploratory. It may be interesting and deserve a proper test. Yet, it hasn’t been given a proper test.
Groups Of Metrics
The authors divide up the measures used into a few groups. Customer Mindset, Customer Behavior and Customer Level Performance metrics are part of Operational Performance. These eventually feed Product Market Performance (e.g., unit sales, new product success). There are also Accounting Performance measures (taken from the financial accounts). Another set of measures are Financial-Market Performance measures. The latter include such things as investor returns, equity risk, and cost of capital.
The sheer variety of performance measures used is exciting — marketing seems connected with a lot of outcomes. That said, it is also highly problematic. Often researchers don’t have a great theory as to what marketing actions should impact. They just trawl for impact. This is a great, if somewhat depressing, paper.
For more on marketing measurement see my popular marketing metrics section here.
Read: Constantine S. Katsikeas, Neil A. Morgan, Leonidas C. Leonidou, and G. Tomas M. Hult. “Assessing performance outcomes in marketing.” Journal of Marketing 80, no. 2 (2016): 1-20.