Metric
|
JC sounds off on Management™
|
When everything about a people is for the time growing weak and ineffective, it begins to talk about efficiency. ... Vigorous organisms talk not about their processes, but their aims.
- — G. K. Chesterton, Heretics
“When a measure becomes a target, it ceases to be a good measure.”
The stock-in-trade of a middle manager and the management consultant she aspires to become.
Like a key performance indicator, a second-order derivative of actual performance calculated to allow non-experts to make cavalier management decisions, usually to reduce expenditure on — aka make redundant — the person performing that function.
To be contrasted with the ineffable, inarticulable skills that are provided by a subject matter expert.
Goodhart’s law
Not a law of economics or sociology so much as a wry remark — professor Goodhart made it at a symposium in 1975 — that happens to pierce modern management orthodoxy through its heart. Thus it can both spur its own industry of academic work in sociology and systems theory, and at the same time go ignored in the upper tiers of corporate management:
When a measure becomes a target, it ceases to be a good measure.
On dolphins, seagulls and opposable thumbs
Goodhart’s law is universal enough to apply even to dolphins.
An aquarium in Miami is reported to have dealt with the problem of litter and dead seagulls in the main tank by rewarding the resident dolphins for cleaning them up: a fish for each bird or piece of litter.
Before long, the dolphins were observed shredding bits of litter into smaller pieces and claiming multiple fish, and then stockpiling surplus fish, luring seagulls with them, and killing the seagulls.[1]
“We should ” as someone on the podcast remarked, “be grateful dolphins don’t have opposable thumbs.”
Four problems with metrics
One could, and here I am indebted to this excellent resource on Goodhart’s law, break the phenomenon down into four components.
Regressive
Using a single metric as a proxy to measure “multivariate” phenomena that are driven by several factors. Here Simpson’s paradox is not your friend. Much “social justice” — which we define as the wishful, if not wilful, tendency to boil complex socioeconomic phenomena down to simplistic moral propositions that even a dull fifth-former could understand, and only a dull fifth-former would fall for — stumbles into this trap.
Extremal
Where a given metric is useful within a range — such a range generally corresponds to “normalcy”: “peacetime”, “normal operating conditions”, “business as usual” and similar platitudes — but which breaks down, fails, or even reverses itself in extremes or unusual cases beyond that range. Paging Messrs Black and Scholes. You know, using normal distributions of independent events to model dependent events, like human behaviour of the market. These are especially fraught because 80% of the time these metrics work, and they work fabulously, is exactly the range over which it doesn’t matter whether they work or not. When things aren’t blowing up. The use case for the metric in the first place, remember, was to warn you about risk events. A “heightened risk” metric that you can only rely on when there isn’t a heightened risk is a waste of cheese.
Causal
The old correlation is not the same as causation chestnut. It may be true that people often buy ice-cream when they are wearing sunglasses, but handing out complimentary sunglasses will not improve ice-cream sales.
Adversarial
Substitute targeting the desired outcome — a senior tranche in a portfolio of mortgages that will not default in any circumstances — with one that is rated AAA. Hello, global financial crisis.
See also
- Outsourcing
- Service level agreement
- Management consultant
- Subject matter expert
- Metrics
- ClauseHub: theory
- Beware of shorthand
- Multivariate factors
References
- ↑ [https://open.spotify.com/episode/3y799K1qGOhqxPUGcqXwx0 Rationally Speaking podcast episode 240.