Jerry Z. Muller

In a nutshell

We increasingly live in a culture of metric fixation: the belief of so many organizations that scientific management means replacing judgment based upon experience and talent with standardized measures of performance, and then rewarding or punishing individuals and organizations based upon those measures. The buzzwords of metric fixation are all around us: “metrics,” “accountability,” “assessment,” and “transparency.”

The Tyranny of Metrics treats metric fixation as the organizational equivalent of The Emperor’s New Clothes. Though often characterized as “best practice,” metric fixation is in fact often counterproductive, with costs to individual satisfaction with work, organizational effectiveness, and economic growth. The book helps explain why metric fixation has become so popular, why it is so often counterproductive, and why some people have an interest in promoting it. It is a book that analyzes and critiques a dominant fashion in contemporary organizational culture, with an eye to making life in organizations more satisfying and productive. It’s a book about management broadly construed. But unlike most authors who write about management, it also tries to see organizations from the perspective of the managed.

A close-up

Based on the real-life experiences of its creators, David Simon and Ed Burns, the HBO series “The Wire,” is regarded by some as among the greatest cultural documents of our age. And with good reason. Focused on a single American city, Baltimore, the series drills down into a few major institutions – the police, the school system, municipal politics, the press – and provides an X-ray-like image of their workings and dysfunctions. The series has attracted an international audience because its themes of organizational dysfunction resonate broadly across Western societies.

One of the recurrent themes of “The Wire” is the salience of metrics: of measured performance as the hallmark of “accountability.” Police commanders are obsessed with hitting the numbers, and they do so by a variety of means that sacrifice effectiveness to statistical targets. Politicians demand numbers that attest to police success in controlling crime. So the police units do their best to avoid having murders attributed to their district: when it turns out that a drug gang has been disposing of bodies in abandoned houses, the homicide sergeant discourages their discovery, since that would diminish the “clearance rate,” the metric of the percentage of crimes solved. Much of the plot revolves around dedicated detectives seeking to develop a complex criminal case against a major drug lord. But since building that case will take months if not years, they are discouraged from doing so by the higher-ups, who want the cops to rack up favorable metrics by arresting lots of low-level drug dealers, despite the fact that those arrested will be replaced almost instantly. The major’s office demands that the rate of major crimes will decline by five percent before the end of the year, a target that can only be reached by overlooking actual crimes or downgrading their seriousness. In each case, they are engaged in “juking the stats” – improving their metrics either by distorting actual results, or by diverting their time and effort from more productive to less productive uses.

Another plot line involves an ex-cop who teaches in a middle school in a neighborhood plagued by poverty, drug abuse, and family fragmentation. Students in the school perform poorly, and the school is in danger of being closed if the test scores of its students do not improve. So, in the six weeks before the standardized tests of English are to be administered, the teachers are instructed by their principal to focus all of class time on practicing for the tests, ignoring other subjects entirely (euphemistically denoted “curriculum alignment”). “Teaching to the test,” like juking the stats, is a way in which institutions are perverted, as effort is diverted from the institution’s true purpose (education) to meeting the metric targets on which its survival has come to depend.

The distortive effects of performance metrics are felt at least as much across the Atlantic, in Great Britain. There, another television series penned by a former real-life practitioner captures the same phenomenon. The series, “Bodies,” written by Jed Mercurio, a former hospital physician, takes place in the obstetrics and gynecology ward of a metropolitan hospital. In the first episode, a newly arrived senior surgeon performs an operation on a patient with complex co-morbidities, after which she dies. His rival then provides him with this advice: “The superior surgeon uses his superior judgment to steer clear of any situation that might test his superior ability.” That is, he avoids difficult cases as a way of maintaining his success rate. A classic strategy of “creaming,” that is, avoiding risky instances that might have a negative impact on one’s measured performance. The cost of this is that patients at greater risk for a failed surgery are left to a certain death without surgery.

“Bodies” is a medical drama, but the phenomena it depicts exist in the real world. Numerous studies have shown that when surgeons, for example, are rated or remunerated according to their success rates, some respond by refusing to operate on patients with more complex or critical conditions. Excluding the more difficult cases – those that involve the likelihood of poorer outcomes – improves the surgeons’ success rates, and hence their metrics, their reputation, and their remuneration. That of course comes at the expense of the excluded patients, who pay with their lives. But those deaths don’t show up in the metrics.

As readers will see, gaming the metrics occurs in every realm: in policing; in primary, secondary and higher education; in medicine; in non-profits; and, of course, in business. And gaming is only one class of problems that inevitably arise when using performance metrics as the basis of reward or sanction. There are things that can be measured. There are things that are worth measuring. But what can be measured is not always what is worth measuring. What gets measured may have no relationship to what we really want to know. The costs of measuring may be greater than their benefits. The things that get measured may draw effort away from the things we really care about. And measurement may provide us with distorted knowledge – knowledge that seems solid but is actually deceptive.

Lastly

My purpose is in part to help articulate the vague sense of frustration that so many people have with the organizations in which they work. By pointing out the problems with metric fixation in a systematic way, I hope to make it possible for people to explain to themselves what has gone wrong, and to provide some useful guidelines about when to use measurement and how to do so. My point is not that measurement is bad. It is that measurement requires judgment: about what is worth measuring, about the costs of measuring, about the significance of what is measured.

Recently, I gave a talk about the book at a local Washington bookstore. During the Q&A, a fellow who had recently retired from a career in foreign aid rose and said that the dysfunctions I described were exactly what he had experienced in recent years. After the talk, he bought ten copies of the book, to distribute to his former bosses in foreign aid organizations. That’s the sort of reaction I’m hoping for.