Metric: Difference between revisions

Metric (view source)

Revision as of 09:16, 2 May 2023

153 bytes removed , 2 May 2023

no edit summary

Amwelladmin

Bureaucrats, Interface administrators, Administrators

82,891

edits

@@ Line 12: / Line 12: @@
 An excellent page of resources on [[Goodhart’s law]] to be found [https://modelthinkers.com/mental-model/goodharts-law here].
-Regressive
+*'''Regressive''': using a single metric as a proxy to measure a phenomenon that is actually [[multivariate]] — caused by several factors. Here [[Simpson ’s paradox]] is not your friend.
-Choosing a measurement that is a single proxy to your goal, when the goal actually has several causal factors behind it.
+*'''Extremal''': Where a the metric is a useful indicator under normal circumstances — [[Mediocristan]], but breaks down in extremes or unusual cases — [[Extremistan]]. Paging Messrs [[Black Scholes option pricing model|Black and Scholes]]. You know, using [[normal distribution]]s of independent events to model dependent events, like human behaviour of the market.
-Knowing that successful basketballers tend to be tall, you might build a basketball team by recruiting tall people. However, they might not possess other factors such as coordination, fitness, quick responses etc..
+*'''Causal''': the old [[correlation]] is not the same as [[causation]] chestnut.
-Extremal
+*'''Adversarial''': Substitute targeting the desired outcome — a senior tranche in a portfolio of mortgages that will not default in any circumstances — with one that is rated AAA. Hello, [[global financial crisis]].
-Where a proxy might indicate the goal under normal circumstances, but not in variable contexts.
-Sugar consumption helped define our ancient ancestors’ survival however today, with a different context, our excessive sugar consumption is often an indicator of an unhealthy diet.
-Causal
-Confusing Causation vs Correlation when selecting measurements.
-Data might indicate increasing ice cream sales when people wear sunglasses. However, providing more sunglasses won’t necessarily boost ice cream sales because they are both caused by a third factor, ie. sunny, hot days.
-Adversarial (Cobra Effect)
-Choosing a proxy measurement incentivises people to make the proxy the new goal.
 {{Outsourcing}}
 *[[Beware of shorthand]]
+*[[Multivariate]] factors