Template:Data as a self-fulfilling prophecy

Knowledge is knowing that a tomato is a fruit; wisdom is not putting it in a fruit salad.

—Miles Kington

There is, the JC freely narratises, an epochal battle raging between wisdom and technocracy which the technocrats have, for thirty years, been winning. As we are gradually immersed in the superficial charm of technology, it feels like an end-game: there is no way out for the meatware: no titanic clash, no great final conflict — just a feeble whimpering out of human expertise, finally beaten down by the irrepressible energy of the algorithm. The latest front, artificial intelligence, feels like a coup de grace the inevitable endpoint of human uselessness.

Weisendämmerung: the twilight of the wise.

Wisdom only comes with time, experience and anecdotally-accumulated expertise. It is hard to acquire and expensive to buy. Technology, by contrast requires no time and no expertise: only brute information processing capacity — which gets ever cheaper — and enough data to process — which gets ever more abundant.

The more data there is, the more you can process, the more neural networks can crawl over it, pattern matching and framing and analysing and the more “insight” — unexpected, machine-generated insight — we can extract.

Observation of “data” is theory-dependent

But this insight is a function of the data: we can’t analyse or pattern-match data we don’t have — and what we do have we must select, filter, format, array and frame according to some pre-existing theory of the game. Our data paints a picture from shadows: by blocking out “irrelevant” data we have collected but which doesn’t advance, bear upon or fit our theory. The more data we have, the more of it we must block to make a meaningful model.

So, dilemma: the less data we have, the stronger the model, but the less reliable the insight, because we don’t know what we’re missing. The more data, the weaker the model, and the less reliable the insight, because we still don’t know what we’re missing, but the more of what we do know we have had to rule out to draw a single coherent model.

Data proves nothing in the abstract. It can be made to prove anything in the particular.

How experts work

We don’t know how experts do what they do. That ineffability is their very expertise, since if we did know, we wouldn’t need them. As experts increasingly use digital tools, though, they spin off more and more data that the technocrats can collect and analyse. (A conversation across the desk is purely analogue; it contains no recordable data or metadata; a typed letter is an analogue artefact with no metadata; a facsimile is a digital graphic of an analogue artefact with limited extractable data or metadata; an electronically transmitted ASCII document is only data, and has no meaningful analogue existence at all).

How technocrats work

As we succumb to data, increasingly giving it off, great clods of it, which the technocrats then harvest and weaponise back at us in some self-fulfilling apocalyptic prophecy. Because they can measure, they do measure. No matter what they measure is meaningless, or that the data are necessarily historical: a formalistic digital sketch of a model; they can only see what they can see: they cannot measure of the value of actions not taken, crises headed off; investment costs avoided through quick thinking and untraced application of human common sense, because necessarily, there is no data about did not happen.

We design digital tools to make lives easier. They happen, as a by-product to generate metadata. Though the metadata was not the reason for the tool, it becomes the justification for it. When the ineffable magic has happened, and evaporated into the atmosphere, the metadata is all the residue that remains, not the lives made easier by the magic.