The Tyranny of Metrics – Jerry Z Muller

Reading a variety of books provides you with different perspectives. The Tyranny of Metrics by Jerry Z Muller is of that kind. We have heard people say data is the new oil, metrics are critical and every decision being made should be based on data. However, if we over-emphasize data, we may not have doctors and engineers take more risks. 

If a doctor is obsessed with the success rate of his surgeries he will not take on riskier surgeries (and this will end up with more people dying). The same is the case with universities or schools where they are focussed on the grades of the students (and the overall quality dips in the long run). 

Common metric mistakes

The tyranny of Metrics talks about the common errors made with the emphasis on metrics. 

1. Measuring the most easily measurable (this usually does not help)
2. Measuring the simple when the desired outcome is complex
3. Measuring the inputs rather than the outcomes (The number of resources working on a project as opposed to the outcome of the work)
4. Distortion of data.

Why do people rely on data?

The over-reliance on data happens because of social mistrust. In large organizations/ universities, being reliant on data is the short-cut to making decisions. Also, in public organizations, if decisions are made based on data rather than human thinking the litigations are less likely. 

Universities

The universities are ranked based on different metrics. However, there is a great possibility for the metrics to be flawed. Also, incentivizing metrics does not result in the desired outcomes. For example, if the rankings focus on the number of research papers published, universities tend to increase the number of papers published and not the quality of the research paper. 

Pay for Performance

Corporations work on Pay for Performance and it is difficult to do the same in Government organizations, hospitals, universities, and more. Doctors may not be motivated enough to treat the elderly. This may also end up harming patient care. For example, if readmission is a metric being measured, some hospitals put the patient on observation and treat them as an outpatient. One would need to check if the system is being gamed. All things need to be considered when using metrics to arrive at decisions. 

Policing

Measuring crime rates could also result in underreporting the crime by the police. Developing valid metrics to measure success and failure also requires local knowledge. 

Business and Finance

Investing for the long term should always be encouraged. Pay for Performance also resulted in the 2008 financial crises where the system was gamed. The Wells Fargo scandal in 2011, where the Well Fargo associates started opening dubious accounts. Mylan, where they kept raising the prices of an essential drug (Epipen). The price of the Mylan stock kept rising, however, people had to pay more to buy drugs. The stock market driven by quarterly results does not aid long term thinking or does not incentivize innovation. The metrics followed by the stock market – Does that even make sense?

The Tyranny of Metrics ends with When and How to use metrics? You also need the buy-in from everyone for the metrics to be successful. It also needs to come from the bottom. Do not measure the obvious, have local knowledge, and do not over-rely on metrics are some of the key learnings from the book.