How the laws of data science lie

The design of organisations and products The Jolly Contrarian holds forth™ →Go Premium File:Chatgptliar.jpg Resources and Navigation Making legal contracts a better experience Index — Click ᐅ to expand: Design Index: Click ᐅ to expand: Essays

edit

When a man throws a ball high in the air and catches it again, he behaves as if he had solved a set of differential equations in predicting the trajectory of the ball. He may neither know nor care what a differential equation is, but this does not affect his skill with the ball. At some subconscious level, something functionally equivalent to the mathematical calculations is going on.

—Richard Dawkins

Really powerful explanatory laws of the sort found in theoretical physics do not state the truth.

—Nancy Cartwright, How the Laws of Physics Lie (1983)

In 1983 philosopher Nancy Cartwright wrote the seemingly scandalous book How the Laws of Physics Lie. It is not quite the post-modernist screed it sounds, but rather a serious and literate, but difficult, work of analytical philosophy. Cartwright’s argument is that scientific laws are formulated in conditions so rigid, isolated and controlled that, even though they might be perfectly valid within those conditions, they are practically useless “in the real world,” where those conditions have no hope of existing.

We explain by ceteris paribus laws, by composition of causes, and by approximations that improve on what the fundamental laws dictate. In all of these cases the fundamental laws patently do not get the facts right.

So Newton’s mechanics assuming, as they do, no inconveniently intervening forces like friction, gravity, inelasticity, might plot the trajectory of an object on a graph, but have no chance with the proverbial crisp packet blowing across St Mark’s Square. You could spend a lot of time with a slide rule and an anemometer; when you look up the packet will be gone.

The same observation animates Gerd Gigerenzer’s faith in heuristics over science: despite Richard Dawkins’ trite conviction to the contrary, a fielder performs no differential equations on the way to catching a flying cricket ball.

We trick ourselves into believing the power of our scientific laws, wilfully blind to the ad hoc variations, adjustments and glosses that our messy world imposes upon them; we put down any apparent disparity to this ineffable collection of intervening forces: we tell ourselves the laws of physics describe an idealised, Platonic model; our messy world is anything but, so we should expect variances from those pure predictions.

Now this is all well and good: we are simply “pragmatising” scientific laws: recasting them as rules of thumb and generalised guides to how the physical world will behave — they set outer bounds to our expectations but will not give us a fine-grained real-time means of navigating the world. For example: the laws of physics tell us no cricket ball will attain escape velocity from a human bat-swing, however well struck; nor will it morph into a bowl of petunias as we try to catch it. But as for the precise trajectory of spacetime it will pass through as I try to catch it — if science could ever yield an approximation of that, it would be far too late to be useful, and really we don’t have anything like enough data to run the calculations in any case.

So we need to rely on judgment, the facility for catching we have acquired through a lifetime of experience and practice, none of which involved solving a single differential equation: you cannot, as Nassim Taleb says, lecture birds on how to fly. Science is a convenient post-hoc explanation of what we did, not a guide to what we must do.

But the physical world is a complicated system: generally, a very, very complicated system, but insofar as the law of physics are concerned, not a complex one: we do not, by applying our models of physics to to the world, change how the world behaves. It is still in a sense linear: it is just our rules are approximations, not specific predictions. So the “lie” perpetrated by the laws of physics is broadly benign.

The JC’s sense is a similar thing may be true of data science, only it is less benign.

We tell ourselves that data models can predict our behaviour, are unfailingly accurate, that we should yield to their greater power. We no longer need “thick” human rules of moral principle to moderate our behaviour, because machines can systematically apply infinitesimally thin rules that equably adjudicate on any given particular. This is all the more concerning with the advent of neural networks and large language models that we readily confess we do not understand at all, but we were already there, in our collective obeisance to, for example, the truth of DNA testing, or GPS navigation, or automated self-triage. It seems plausible; we don’t feel like we have good grounds to challenge it, so we defer to it. We suppose spitting in a tube can tell us with certainty that we are 99.4% Scottish, 0.2% North African with a smudge around Scandinavia, less than 4% Neanderthal, but don’t pick up any African heritage at all — despite the fact that every human on the planet is, ultimately, 100% African by origin (homo sapiens diverged from homo neanderthalensis hundreds of thousands of years before any human departed Africa).

These thin rules lie: they give us a false comfort in the truth of the things they opine about, the same way science does. So there aren’t really 590 calories in that burger — it seems plausible if it is printed on the menu card, but the more permanently it is printed the less likely it is to be true. There are not really 49.57km in those directions to the airport, the DNA tests really don’t know whether you are partly Bulgarian, ten thousand steps won’t transport you to health, ten thousand hours won’t make you into a concert violinist, two litres of water won’t ward off dehydration — but you as a layperson and none the wiser, so the claim can be made and got away with. It's not independently testable. How would you know? Your implicit trust in untestable propositions but gets trust, and from nowhere the data modernists have bootstrapped themselves into a kind of credibility.

Premium content

Here the free bit runs out. Subscribers click 👉 here. New readers sign up 👉 here and, for ½ a weekly 🍺 go full ninja about all these juicy topics 👇

edit

See also

edit