Template:M intro technology rumours of our demise: Difference between revisions

So, about that “[[division of labour]]”. When it comes to mechanical tasks of the “body”, [[Turing machine]]s scale well, and humans scale very badly. “Scaling”, when we are talking about computational tasks, means doing them over and over again, in series or parallel, quickly and accurately. Each operation can be identical; their combined effect astronomical. Of course machines are good at this: this is why we build them. They are digital: they preserve information indefinitely, however many processors we use, with almost no loss of fidelity.

You could try to use networked humans to replicate a [[Turing machine]], but the results would be slow, costly, disappointing and the humans would not enjoy it.<ref>{{author|Cixin Liu}} runs exactly this thought experiment in his ''Three Body Problem'' trilogy.</ref> Humans are slow and analogue. With each touch the [[signal-to-noise ratio]] would quickly degrade: this is the premise for the parlour game “Chinese Whispers” — with each repetition  the signal degrades, with amusing consequences.   

But when it comes to “mind stuff,” humans scale well. “Scaling”, for imaginative tasks, is different. Here we don’t want identical, digital, high-fidelity ''duplication'': ten thousand copies of ''Finnegans Wake'' contribute no more to the human canon than does one.<ref>Or possibly, even ''none'': Wikipedia tells us that, “due to its linguistic experiments, stream of consciousness writing style, literary allusions, free dream associations, and abandonment of narrative conventions, ''Finnegans Wake'' has been agreed to be a work largely unread by the general public.”</ref> Multiple humans doing “mind stuff” contribute precisely that idiosyncrasy, variability, difference in perspective that generates the wisdom, or brutality, of crowds: a complex community of readers can independently parse, analyse, explain, narratise, extend, criticise, extrapolate, filter, amend, correct, and improvise the information ''and each others’ reactions to it''.  

The wisdom of the crowd shapes itself: consensus has a directed intelligence all of its own. This community of expertise is what [[Sam Bankman-Fried]] misses in his dismissal of Shakespeare’s “[[Bayesian prior|Bayesian priors]]”. However handsome William Shakespeare’s own contribution to the Shakespeare canon — the actual folio — it is finite, small and historic. It was complete by 1616 and hasn’t been changed since. The rest of the body of work we think of as “Shakespeare” comprises interpretations, editions, performances, literary criticisms, essays, adaptations and its (and their) infusion into the vernacular and it ''vastly outweighs the master’s actual folio''. What it more, it continues to grow.

This kind of “communal mind scaling” creates its own intellectual energy and momentum from a small, wondrous seed planted and nurtured four hundred years ago.<ref>On this view Shakespeare is rather like a “prime mover” who created a universe but has left it to develop according to its own devices.</ref>  No matter how fast pattern-matching machines run in parallel, however much brute, replicating horsepower they throw at the task, it is hard to artificial intelligence in the shape of a large learning model, without human steering, doing any of this.

(The “directed intelligence of human consensus” is not [[utopia|magically benign]], of course, as [[Sam Bankman-Fried]] might be able to tell us, having been on both ends of it).<ref>See also Lindy Chamberlain, Peter Ellis and the sub-postmasters wrongly convicted in the horizon debâcle.</ref>

Last point on literary theory, is that the “[[Bayesian priors]]” argument which fails for Shakespeare fails all the more so for a [[large language model]].  

Just as a great deal of the intellectual energy involved in rendering a text into the three-dimensional [[metaphor]]ical universe we think of as ''King Lear'' comes from the cultural milieu surrounding the text, and not its actual author, so it does with the output of an LLM. Its its source after all, is entirely drawn from the human canon. A large language model trained only on randomly assembled ASCII characters would return randomly assembled ASCII characters.

But what if the material is not random? What if the model augments its training data with its own output? Might that create an apocalyptic feedback loop whereby LLMs bootstrap themselves into some kind of hyperintelligent superlanguage, beyond the cognitive capacity of an unaugmented human, from which platform they might dominate human discourse? Are we inadvertently seeding ''Skynet''?

Just look what happened with AlphaGo. It didn't require ''any'' human training data. They just fed it the rules, switched it on and, with indecent brevity, it had walloped the game’s ruling grandmaster. It learned by playing millions of games against itself.  What happens when LLMs do that?

Brute-forcing outcomes in a fully bounded, zero-sum environment with simple, fixed rules is exactly what machines are designed to do. If this is to be our comparison, we are misdirecting ourselves, voluntarily suspending disbelief, setting ourselves up to fail. This is agreeing a contest on the machine’s terms. For playing Go is “body stuff”. The environment is fully fixed, understood, transparent and determined. There is no uncertainty. No ambiguity. No rom for creative construal. ''This is exactly where we would expect a Turing machine to excel''.

We deploy LLMs by contrast, in complex environments. They are unbounded, ambiguous, inchoate and impermanent. The situation changes. Mathematical computation doesn't work.

{{Quote|Imagine how hard physics would be if particles could think.

An LLM works because it can quickly composite synthetic outputs ''against human text''. It must pattern-match against well-formed human language. That is how it works its magic.  Degrading its training data will progressively degrade its output. This “model collapse” is an observed effect. LLMs will only work for humans if they’re fed human generated content. A redditor put it more succinctly than I can:

There is another contributor to the cultural milieu surrounding any text: the ''reader''. It is the reader, and her “[[cultural baggage]]”, who must make head and tail of the text. She alone determines, for her own case, whether it stands or falls. This is true however rich the cultural milieu that supports the art.

 

Construing natural language, much less visuals or sound, is no matter of mere [[Symbol processing|symbol-processing]]. Humans are ''not'' [[Turing machine|Turing machines]].  

We know this because the overture from ''Tristan und Isolde'' can reduce different listeners to tears of joy or boredom. One contrarian can see in the Camden Cat a true inheritor of the great blues pioneers, while others see an unremarkable busker. A text sparks meaning, and becomes art, in the reader’s head.  

It turns out that hitherto sterile debate about the nature of meaning and art that turned generations of students off philosophy is now critically important.

If you feed an LLM with its own output it rapidly degrades into meaningless mush. LLMs are not intentional. They are not directed, they depend on the ongoing environment to shape their fitness. That environment is necessarily human.