Standard Deviations
Gary Smith’s impressive Standard Deviations book concerns an important point. Statistically inclined people often seem to miss that theory small t matters. Smith is keen to note that data isn’t enough on its own. “Data without theory… is treacherous” (Smith, 2014, page 233).
Smith describes a case of a cholera outbreak. This outbreak was statistically associated with people not leaving their villages a few days before. One might use this insight to look for lack of movement between villages. We might conclude that we can predict cholera from movements of people. This analysis is a lot of work to go through. It is also unnecessary when a simple piece of “theory” is better.
By theory I mean simple thought about causes. Theory (small t) helps us work out what is happening. Theory helps us do this without massive amounts of number crunching. What is happening is pretty simple. The explanation is this, floods come and people stop leaving their villages. Cholera comes soon after the floods borne by the floodwater. We can predict cholera from easily observed floods. As such, we don’t need to capture movement data of people to make this prediction.
Having A Causal Theory Is Vital To Understanding The World
Thinking — developing a causal theory — allows us to use the data much more effectively. Theory without data is a problem which can afflict academics . The obvious risk of theory without data is that we can get divorced from reality.
Small t Theory Matter
That said, data without theory is also a major problem. it is the sort of problem that managers might not see as clearly. Basing your idea purely on data can leave you believing and doing some pretty silly things. Using data on its own without theory can fuel bubbles . We don’t know why the price is so high but it keeps on going up so we assume the price will continue to do so. My advice is always aim to come up with a at least plausible theory of what is causing whatever you observe in the data. You should always have a plausible story, what I call small t theory, before putting too much faith in any result.
For more see https://neilbendle.com/popular-marketing-metrics/value-of-a-like-do-not-use-for-budgeting/(opens in a new tab)
Read: Gary Smith, 2014, Standard Deviations: Flawed Assumptions, Tortured Data and Other Ways to Lie With Statistics, The Overlook Press.