Metric: Difference between revisions

{{a|mgmt|{{image|Dophin Thumbs Up|jpg|So long, and thanks for all the fish}}}}{{quote|When everything about a people is for the time growing weak and ineffective, it begins to talk about efficiency. ... Vigorous organisms talk not about their [[process]]es, but their aims.

Not a law of economics or sociology so much as a wry remark — professor Goodhart made it at a symposium in 1975 — that happens to pierce modern management orthodoxy through its heart.  Thus it can both spur its own industry of academic work in sociology and [[systems theory]], and at the same time go ignored in the upper tiers of corporate management:

{{Quote|When a measure becomes a target, it ceases to be a good measure.}}People are smart and selfish. They will work any target you set to suit themselves. If you tax by number of windows, people will board up their windows.<ref>See James C. Scott’s epic {{Br|Seeing Like a State}}.</ref>

Goodhart’s law is universal enough to apply even to dolphins.  

An aquarium in Miami is reported to have dealt with the problem of litter and dead seagulls in the main tank by rewarding the resident dolphins for cleaning them up: a fish for each bird or piece of litter.

Before long, the dolphins were observed shredding bits of litter into smaller pieces and claiming multiple fish, and then stockpiling surplus fish, luring seagulls with them, and ''killing the seagulls''.<ref>[https://open.spotify.com/episode/3y799K1qGOhqxPUGcqXwx0 Rationally Speaking podcast episode 240.</ref>

“We should ” as someone on the podcast remarked, “be grateful dolphins don’t have opposable thumbs.”

One could, and here I am indebted to [https://modelthinkers.com/mental-model/goodharts-law this] excellent resource on [[Goodhart’s law]], break the phenomenon down into four components.

Using a single metric as a proxy to measure “[[multivariate]]” phenomena that are driven by several factors. Here [[Simpson’s paradox]] is not your friend. Much “social justice” — which we define as the wishful, if not wilful, tendency to boil complex socioeconomic phenomena down to simplistic moral propositions that even a dull fifth-former could understand, and only a dull fifth-former would fall for — stumbles into this trap.  

Many metrics are useful ''within a range'' corresponding to “normalcy”: call it “peacetime”, “normal operating conditions”, “[[business as usual]]” and similar platitudes — but break down, fail, or even reverse themselves in extremes or unusual cases beyond that range. Using [[normal distribution]]s of independent events to model non-dependent events with non-linear distributions — like, well, anything that involves human behaviour, such as a market — is especially fraught, because even where, 95% of the time, your metrics work fabulously, that 95% is exactly the range over which ''it doesn’t matter whether they work or not''. This is the time where things are operating as normal, behaving themselves, and ''not'' blowing up.  

The “use-case” for any metric in the first place, remember, is to warn about risk events. No-one needs a light on the dashboard saying “everything is fine”.  A “heightened risk” metric that you can only rely on when there isn’t a heightened risk is a ''waste of trees''.

 

 

====Adversarial====

Substitute targeting the desired outcome — a senior tranche in a portfolio of mortgages that will not default in any circumstances — with one that is rated AAA. Hello, [[global financial crisis]].