I have previously written about how disappointed I was by Jerry Muller’s The Tyranny of Metrics, see here and here. Today I will look at a broader problem that his book exhibits. Namely ‘The Pseudo Profound Statement’. I see this in a lot of places not just in Muller’s work.
Be Careful With A Pseudo Profound Statement
One should always be careful about the sort of statement that sounds like it means something but really doesn’t.
The problem is not measurement, but excessive measurement and inappropriate measurement — not metrics, but metrics fixation.
Muller, 2018, page 4
Let us ignore the fact that his book’s title is The Tyranny of Metrics not The Tyranny of Metrics Fixation. An editor would almost certainly suggest the shorter, punchier title even though it undermines everything that Muller wanted to say given he is apparently not against metrics despite them being tyrannical.
Instead, let us accept that the statement gets to the heart of what Muller wants to say. Some may agree with Muller. If you are nodding along I totally understand.
Indeed, when I say ‘some may agree’ I can be more specific. 100% of reasonable people would be in agreement with Muller at this point. Who could possibly be for ‘excessive measurement and inappropriate measurement’?
Other Pseudo Profound Things We Are Against
Lets try some other statements in a similar view:
- The problem is not use of consumer data, but excessive use of consumer data and inappropriate use of consumer data — not use of consumer data, but use of consumer data fixation.
I agree with that — who doesn’t?
The problem I have is the pseudo-profound statement is that the argument is meaningless. You can just change ‘metrics’ to almost anything and it still works. How about something that all sides of politics can agree upon.
- The problem is not regulation, but excessive regulation and inappropriate regulation — not regulation, but regulation fixation.
It sounds like a right-wing argument and you can certainly imagine someone on the right saying it. Still, it is mere puffery because proponents of more regulation can also agree. Those arguing for increased regulation might see the need for more regulation. They can agree with the statement and still argue that more regulation would help. They just don’t want inappropriate regulation. Who does? We can all agree on not wanting inappropriate things. The devil in the detail is defining what is inappropriate. A meaningful contribution to the discussion tells us what inappropriate means.
Let Me Go On Record I Am Not Against Water But…
Why not make it broader? Let us take something vital to life and Muller’s statement still works.
- The problem is not water, but excessive water and inappropriate water — not water, but water fixation.
It is a great way of constructing an argument. You can never be wrong. Bit of a waste of everyone’s time though.
Metrics Fixation
To be fair Muller does give a bit more detail on metric fixation. He is against:
- Replacing judgment with metrics. I agree, metric use requires judgment but he gives few specifics on how to use judgment or how to combine metrics and judgment.
- Thinking that data transparency assures institutions carry out their purpose. This is a strange straw man. Who thinks data transparency is magical? I agree anyone who thinks that transparency is a panacea is wrong. That said, transparency can often be useful so lets talk seriously about when and how.
- Thinking that motivation is only from financial rewards tied to measurement. I have some sympathy with this argument. My feeling is that performance related pay is overused but still we need more detail on what Muller wants. Especially given he seems fine with promotion and pay raises as long as they aren’t based upon metrics. Where selective rewards exist it always leads back to the same problem. How do you determine who deserves the reward? This is presumably by using judgment to his mind, but whose judgment?
I think he means the judgment of people like him and transparency of process be damned. Society has a long history of people in power giving rewards to their buddies. Without metrics rewards often simply go to insiders. Using objective metrics certainly isn’t perfect but could potentially mitigate against cronyism. Is not being able to give money to your buddies without any justification really a form of tyranny?
Honestly, I felt a lot of Muller’s arguments lacked any serious attempt to grapple with the big problems that he noted.
So We Learned Metrics Shouldn’t Be Used Except When They Should
I agree. Who wouldn’t?
Read: Jerry Z. Muller (2018) The Tyranny of Metrics, Princeton University Press