The Jolly Contrarian

Template:M intro technology rumours of our demise: Difference between revisions

Template Discussion

Template:M intro technology rumours of our demise (view source)

Revision as of 17:52, 7 December 2023

20,106 bytes added ,  7 December 2023
no edit summary
Tags: Mobile edit Mobile web edit
No edit summary
Tags: Mobile edit Mobile web edit
 
(44 intermediate revisions by the same user not shown)
Line 30: Line 30:
But technology also creates space and capacity to indulge ourselves. [[Parkinson’s law]] states: work expands to fill the time allotted for its completion. Technology ''also'' frees us up to care about things we never used to care about. The microcomputer made generate duplicate and distribute documents far, far easier. There’s that boon and bane, again.
But technology also creates space and capacity to indulge ourselves. [[Parkinson’s law]] states: work expands to fill the time allotted for its completion. Technology ''also'' frees us up to care about things we never used to care about. The microcomputer made generate duplicate and distribute documents far, far easier. There’s that boon and bane, again.


So, before concluding that ''this time'' the machines will put us out of work we must explain ''how''. [[this time is different|Why is this time different]]? What has changed?  
So, before concluding that ''this time'' the machines will put us out of work we must explain ''how''. [[this time is different|Why is this time different]]? What has changed?  


===FAANGs ahoy?===
===FAANGs ahoy?===
Line 86: Line 86:
Thirdly — cool gadget maker that pivots to banking ''and does it well'' has as much chance of maintaining millennial brand loyalty as does a toy factory that moves into dentistry.
Thirdly — cool gadget maker that pivots to banking ''and does it well'' has as much chance of maintaining millennial brand loyalty as does a toy factory that moves into dentistry.


Those Occupy Wall Street gang? Apple fanboys, the lot of them. ''At the moment''. But it isn’t the ''way'' [[trad fi]] banks go about banking that tarnishes your brand. It’s ''banking''. ''No-one likes money-lenders''. It is a dull, painful, risky business. Part of the game is doing shitty things to customers when they lose your money. Repossessing the Tesla. Foreclosing on the condo. That ''isn’t'' part of the game of selling MP3 players.
Those Occupy Wall Street gang? Apple fanboys, the lot of them. ''At the moment''. But it isn’t the ''way'' [[trad fi]] banks go about banking that tarnishes your brand. It’s ''banking''. ''No-one likes money-lenders''. It is a dull, painful, risky business. Part of the game is doing shitty things to customers when they lose your money. Repossessing the Tesla. Foreclosing on the condo. That ''isn’t'' part of the game of selling MP3 players.


''The business of banking will trash the brand.''
''The business of banking will trash the brand.''
Line 118: Line 118:
{{Quote|{{ACC on technology}} }}We are in a machine age.  
{{Quote|{{ACC on technology}} }}We are in a machine age.  


We call it that because machines have proven consistently good at doing things humans are too weak, slow, inconstant or easily bored to be good at: ''mechanical'' things.  
We call it that because machines have proven consistently good at doing things humans are too weak, slow, inconstant or easily bored to do well: ''mechanical'' things.


Machines — ''quelle surprise'' — are ''good'' at mechanical things. But state-of-the-art machines, per Arthur C. Clarke, aren’t magic: it just ''seems'' like it, sometimes.  
But state-of-the-art machines, per Arthur C. Clarke, aren’t magic: it just ''seems'' like it, sometimes. They are a two-dimensional, simplified model of human intelligence. A proxy: a modernist [[simulacrum]]. They are a shorthand way of mimicking a limited sort of sentience, potentially useful in known environments and constrained circumstances.


Yet we have convinced ourselves, and trained our children, that machine-like qualities — strength, speed, consistency, modularity, [[fungibility]] and ''mundanity'' — should be their loftiest aims.  
Yet we have begun to model ourselves upon machines. The most dystopian part of John Cryan’s opening quote was the first part — “''today, we have people doing work like robots''” — because it describes a stupid present reality. We have persuaded ourselves that “being machine-like’’ should be our loftiest aim. But if we are in a footrace where what matters is simply strength, speed, consistency, modularity, [[fungibility]] and ''mundanity'' — humans will surely ''lose''.  


But executing a task with strength, speed, consistency, fungibility and patience are lofty aims ''only if you haven’t got a suitable machine''. If you ''have'' got a machine, use it. Let your people do something more useful.  
But we ''aren’t'' in that foot race. Strength, speed, consistency, fungibility and patience are the loftiest aims ''only where there is no suitable machine''.
 
If you ''have'' got a suitable machine, ''use it'': let your people do something more useful.  


If you haven’t, ''build'' one.
If you haven’t, ''build'' one.


===Body and mind as metaphors===
===Body and mind as metaphors===
We are used to using the “Turing machine” as a [[metaphor]] for “mind”: how about inverting that? How about using “body”, in that dishonourably dualist but yet intuitive [[Descartes|Cartesian]] sense — as a [[metaphor]] for a “Turing machine”  and “mind” for natural intelligence? Might “mind” and “body” in this sense be a practical guiding principle for the [[division of labour]] between human and machine?
We are used to the “[[Turing machine]]” as a [[metaphor]] for “mind” but it is a bad metaphor. It is unambitious. It does not do justice to the human mind.
 
Perhaps we could invert it. We might instead use “body” — in that dishonourably dualist, [[Descartes|Cartesian]] sense — as a [[metaphor]] for a Turing machine, and “mind” for natural human intelligence. “Mind” and “body” in this sense, are a practical guiding principle for the [[division of labour]] between human and machine: what goes to “body”, give to a machine: motor skills; temperature regulation; the pulmonary system; digestion; aspiration — the conscious mind has no business there. There is little it can add. It only gets in the way. There is compelling evidence that when the conscious mind takes over motor skills, things go to hell.<ref>This is the premise of {{author|Daniel Kahneman}}’s {{br|Thinking, Fast and Slow}}, and for that matter, [[Matthew Syed]]’s {{br|Bounce}}.</ref>


What goes to “body”, give to a machine. Motor skills. Temperature regulation. The pulmonary system. Digestion. Aspiration. The conscious mind has no business here. There is little it can add. It only gets in the way. There is compelling evidence that when the conscious mind takes over motor skills, things quickly go to hell.<ref>This is the premise of Daniel Kahneman’s {{br|Thinking: Fast and Slow}}, and for that matter, [[Matthew Syed]]’s {{br|Bounce}}></ref>
But leave interpersonal relationships, communication, perception, [[construction]], decision-making in times of uncertainty, imagination and creation to the mind. ''Leave the machines out of this''. They will only bugger it up. Let them ''report'', by all means. Let them assist: triage the “conscious act” to hive off the mechanical tasks on which it depends.<ref>{{Author|Julian Jaynes}} has a magnificent passage in his book {{br|The Origins of Consciousness in the Breakdown of the Bicameral Mind}} where he steps through all the aspects of consciousness that we assume are conscious, but which are not.


But leave interpersonal relationships, communication, perception, [[construction]], decision-making in times of uncertainty, imagination and creation to the conscious mind. ''Leave the machines out of this''. They will only bugger it up. Let them ''report'', by all means. Let them assist: by all means, triage the “conscious act” to hive off mechanical tasks. Let the machines loose on them. Let them provide, on request, the information the conscious mind needs to make its build its models and make its plans, but ''do not let them intermediate that plan''.
“Consciousness is a much smaller part of our mental life than we are conscious of, because we cannot be conscious of what we are not conscious of. How simple that is to say; how difficult to appreciate!”</ref> Let the machines loose on those mechanical tasks. Let them provide, on request, the information the conscious mind needs to make its build its models and make its plans, but ''do not let them intermediate that plan''.


The challenge is not to automate indiscriminately, but ''judiciously''. To ''optimise'', so the conscious mind is set free of tasks it is not good at, not diverted from its valuable work by formalistic requirements of the machine. This can’t really be done by rote.
The challenge is not to automate indiscriminately, but ''judiciously''. To ''optimise'', so humans are set free of tasks they are not good at, and thereby not diverted from their valuable work by formal process better suited to a machine. This can’t really be done by rote.


Here “machine” carries a wider meaning than “computer”. It encompasses any formalised, preconfigured process. A [[playbook]] is a machine. A policy battery. An approval process.
Here, “machine” carries a wider meaning than “computer”. It encompasses any formalised, preconfigured process. A [[playbook]] is a machine. A policy battery. An approval process.


===AI overreach===
===AI overreach===
The magic of sufficiently advanced technologies, like [[artificial intelligence]], induces us to look too far ahead.  
Nor should we be misdirected by the “magic” of sufficiently advanced technology, like [[artificial intelligence]], to look too far ahead.
 
We take one look at the output of an AI art generator and conclude the highest human intellectual achievements are under siege. However good humans artists may be, they cannot compete with the massively parallel power of LLMs, which can generate billions of images some of which, by accident, will be transcendentally great art. 
 
Not only does reducing art to its “[[Bayesian prior]]s” like this stunningly [[Symbol processing|miss the point]] about art, but it suggests those who would deploy artificial intelligence have their priorities dead wrong. There is no shortage of sublime human expression: quite the opposite. The internet is awash with “content”: there is far more than our collected ears and eyes can take in. 
 
And, here: have some more! 
 
''We don’t need more content''. What we need is ''dross management'' and ''needle-from-haystack'' extraction. Machines ought to be really good at this. 
 
===== Digression: Nietzsche, Blake and the Camden Cat =====
{{Quote|''The Birth of Tragedy'' sold 625 copies in six years; the three parts of ''Thus Spoke Zarathustra'' sold fewer than a hundred copies each. Not until it was too late did his works finally reach a few decisive ears, including Edvard Munch, August Strindberg, and the Danish-Jewish critic Georg Brandes, whose lectures at the University of Copenhagen first introduced Nietzsche’s philosophy to a wider audience.
:—''The Sufferings of Nietzsche'', Los Angeles Review of Books, 2018
The [[Bayesian priors]] are pretty damning. ...When Shakespeare wrote, almost all of Europeans were busy farming, and very few people attended university; few people were even literate — probably as low as about ten million people. By contrast, there are now upwards of a billion literate people in the Western sphere. What are the odds that the greatest writer would have been born in 1564?
:—[[Sam Bankman-Fried|Sam Bankman Fried]]’s “sophomore college blog”}}
[[Sam Bankman-Fried]] had a point here, though not the one he thought. 
 
[[Friedrich Nietzsche]] died in obscurity, as did William Blake and Emily Dickinson. They were lucky that the improbability engine worked its magic for them, even if not in their lifetimes. 
 
But how many undiscovered Nietzsches, Blakes and Dickinsons are there, now sedimented into unreachably deep strata of the human canon?
 
How many ''living'' artists are currently ploughing an under-appreciated furrow, stampeding towards an obscurity a [[large language model]] might save them from, cursing their own immaculate “[[Bayesian priors]]”?
 
(I know of at least one: the [[Camden Cat]], who for thirty years has plied his trade with a beat-up acoustic guitar on the Northern Line, and once wrote and recorded one of the great rockabilly singles of all time. It remains bafflingly unacknowledged. Here it is, on {{Plainlink|1=https://soundcloud.com/thecamdencats/you-carry-on?si=24ececd75c0540faafd470d822971ab7|2=SoundCloud}}.)
 
=== If AI is a cheapest-to-deliver strategy you’re doing it wrong ===
{{quote|
{{D|Cheapest-to-deliver|/ˈʧiːpɪst tuː dɪˈlɪvə/|adj}}
Of the range of possible ways of discharging your [[contract|contractual obligation]] to the letter, the one that will cost you the least and irritate your customer the most should you choose it.}}
 
Imagine having personal [[large language model]]s at our disposal that could pattern-match against our individual reading and listening histories, our engineered prompts, our instructions and the recommendations of like-minded readers. 
 
Our LLM would search through the billions of existing books, plays, films, recordings and artworks, known and unknown that comprise the human ''oeuvre'' but, instead of making its own mashups, it would retrieve existing works that its patterns said would specifically appeal to us? 
 
This is not just the Spotify recommendation algorithm, as occasionally delightful as that is. Any commercial algorithm has its own primary goal to maximise revenue. A certain amount of “customer delight” might be a by-product, but only as far as it intersects with that primary commercial goal. As long as customers are ''just delighted'' ''enough'' to keep listening, the algorithm doesn’t care ''how'' delighted they are.<ref>As with the JC’s school exam grades: anything more than 51% is wasted effort.Try as he might, the JC  was never able to persuade his dear old ''Mutti'' about this.</ref>
 
Commercial algorithms need only follow a ''[[cheapest to deliver]]'' strategy: they “[[satisfice]]”. Being targeted at optimising revenue, they converge upon what is likely to be popular, because that is easier to find. Why scan ocean deeps for human content when you can skim the top and keep the punters happy enough?
 
This, by the way, has been the tragic commons of the collaborative internet: despite [[Chris Anderson]]’s forecast in 2006, that universal interconnectedness would change economics for the better<ref>[[The Long Tail: How Endless Choice is Creating Unlimited Demand]] ''(2006)''</ref> — that, suddenly, it would be costless to service the long tail of global demand, prompting some kind of explosion in cultural diversity — the exact opposite has happened.<ref>It is called “{{Plainlink|https://www.theclassroom.com/cultural-convergence-examples-16778.html|cultural convergence}}”.</ref> The overriding imperatives of [[scale]] have obliterated the subtle appeals of diversity, while sudden, unprecedented global interconnectedness has had the [[system effect]] of ''homogenising demand''.
 
This is the counter-intuitive effect of a “[[cheapest-to-deliver]]” strategy: while it has become ever easier to target the “[[fat head]]”, ''the [[long tail]] has grown thinner''.<ref>{{author|Anita Elberse}}’s [[Blockbusters: Why Big Hits and Big Risks are the Future of the Entertainment Business|''Blockbusters'']] is excellent on this point. </ref> As the tail contracts, the [[commercial imperative]] to target the lowest common denominators inflates. ''This is a highly undesirable feedback loop''. It will homogenise ''us''. ''We'' will become less diverse. ''We'' will become more [[Antifragile|fragile]]. We will resemble machines. ''We are not good at being machines''.
 
Shouldn’t we be more ambitious about what [[artificial intelligence]] could do for ''us''? Isn’t “giving you the bare minimum you’ll take to keep stringing you along” a bit ''underwhelming''? Isn’t using it to “dumb us down” a bit, well, ''dumb''?
 
===Digression: Darwin’s profligate idea===
The theory of [[evolution by natural selection]] really ''is'' magic: it gives a comprehensive account of the origin of life that [[Reductionism|reduces]] to a mindless, repetitive process that we can state in a short sentence:
 
{{Quote|In a population of organisms with individual traits whose offspring inherit those traits only with random variations, those having traits most suited to the prevailing environment will best survive and reproduce over time.<ref>This is the “ugliest man” by which, [[Nietzsche]] claims, we killed God. </ref>}}
 
The economy of [[Design|''design'']] in this process is staggering. The economy of ''effort in execution'' is not. Evolution is ''tremendously'' ''wasteful''. Not just in how it ''does'' adapt, but in how often it ''does not''. For every serendipitous mutation, ''there are millions and millions of duds''.
 
The chain of adaptations from amino acids to Lennon & McCartney may have billions of links in it, but that is a model of parsimony compared with the number of adaptations that ''didn’t'' go anywhere — that arced off into one of design space’s gazillion dead ends and just fizzled out.
 
[[Evolution]] isn’t directed — that is its very super-power — so it fumbles blindly around, fizzing and sparking, and only a vanishingly small proportion of mutations ever do anything ''useful''. Evolution is a random, [[stochastic]] process. It depends on aeons of time and burns unimaginable resources. Evolution solves problems by ''brute force''. 
 
Even though it came about through undirected evolution, “natural” mammalian intelligence, whose apogee is ''homo sapiens'', ''is'' directed. In a way that DNA cannot, humans can hypothesise, remember, learn, and rule out plainly stupid ideas without having to go through the motions of trying them.
 
All mammals can do this to a degree; even retrievers.<ref>Actually come to think of it Lucille, bless her beautiful soul, didn’t seem to do that very often. But still. </ref> Humans happen to be particularly good at it. It took three and a half billion years to get from amino acid to the wheel, but only 6,000 to get from the wheel to the Nvidia RTX 4090 GPU.
 
Now. [[Large language model]]<nowiki/>s are, like evolution, a “brute force”, ''undirected'' method. They can get better by chomping yet more data, faster, in more parallel instances, with batteries of server farms in air-cooled warehouses full of lightning-fast multi-core graphics processors. But this is already starting to get harder. We are bumping up against computational limits, as Moore’s law conks out, and environmental consequences, as the planet does.
 
For the longest time, “computing power” has been the cheap, efficient option. That is ceasing to be true. More processing is not a zero-cost option. We will start to see the opportunity cost of devoting all these resources to something that, at the moment, creates diverting sophomore mashups we don’t need.<ref>We would do well to remember Arthur C. Clarke’s law here. The parallel processing power an LLM requires is already massive. It may be that the cost of expanding it in the way envisioned would be unfeasibly huge — in which case the original “business case” for [[technological redundancy]] falls away. See also the [[simulation hypothesis]]: it may be that the most efficient way of simulating the universe with sufficient granularity to support the simulation hypothesis is ''to actually build and run a universe'' in which case, the hypothesis fails.</ref>
 
We have, lying unused around us, petabytes of human ingenuity, voluntarily donated into the indifferent maw of the internet. ''We are not lacking content''. Surely the best way of using these brilliant new machines is to harness what we already have. The one thing homo sapiens ''doesn’t'' need is ''more unremarkable information''.
 
====LibraryThing as a model====
So, how about using AI to better exploit our existing natural intelligence, rather than imitating or superseding it? Could we, instead, create [[System effect|system effects]] to ''extend'' the long tail?
 
It isn’t hard to imagine how this might work. A rudimentary version exists in LibraryThing’s recommendation engine. It isn’t new or wildly clever '''—''' as far as I know, LibraryThing doesn’t use AI — each user lists, by ISBN, the books in her personal library. The LibraryThing algorithm will tell you with some degree of confidence, based on combined metadata, whether it thinks you will like any other book. Most powerfully, it will compare all the virtual “libraries” on the site and return the most similar user libraries to yours. The attraction of this is not the books you have in common, but the ones you ''don’t''.
 
Browsing doppelganger libraries is like wandering around a library of books you have never read, but which are designed to appeal specifically to you.
 
Note how this role — seeking out delightful new human creativity — satisfies our criteria for the [[division of labour]] in that it is quite beyond the capability of any group of humans to do it, and it would not devalue, much less usurp genuine human intellectual capacity. Rather, it would ''empower'' it.
 
Note also the [[system effect]] it would have: if we held out hope that algorithms were pushing us down the long tail of human creativity, and not shepherding people towards its monetisable head, this would incentivise us all to create more unique and idiosyncratic things.
 
It also would have the system effect of distributing wealth and information — that is, [[strength]], not [[Power structure|power]] — ''down'' the curve of human diversity, rather than concentrating it at the top.
 
==== Division of labour, redux ====
So, about that “[[division of labour]]”. When it comes to mechanical tasks of the “body”, [[Turing machine]]s scale well, and humans scale badly.
 
===== Body scaling =====
“Scaling”, when we are talking about computational tasks, means doing them over and over again, in series or parallel, quickly and accurately. Each operation can be identical; their combined effect astronomical. Nvidia graphics chips are so good for AI because they can do 25,000 ''trillion'' basic operations per second. ''Of course'' machines are good at this: this is why we build them. They are digital: they preserve information indefinitely, however many processors we use, with almost no loss of fidelity.


We see an AI art generator and immediately conclude that the highest order of human intellectual achievement is at stake. Not only does reducing art to “[[Bayesian prior]]s” stunningly [[Symbol processing|miss the point]] about art — in an artificially intelligent way, ironically — but it skips over all the easier, drearier, more machine-like applications to which machines might profitably put, but with which the poor inconstant human is still burdened. For these mundane but potentially life-changing tasks there is, apparently, no technological resolution in sight: machines that can fold washing, remember where you put the car keys, weed out fake news, and wipe down the kitchen table,wipe a baby’s arse.  
You could try to use networked humans to replicate a [[Turing machine]], but the results would be slow, costly, disappointing and the humans would not enjoy it.<ref>{{author|Cixin Liu}} runs exactly this thought experiment in his ''Three Body Problem'' trilogy.</ref> With each touch the [[signal-to-noise ratio]] degrades: this is the premise for the parlour game “Chinese Whispers”.


Okay, these require motor control and interaction with the irreducibly messy [[off-world|real world]], so there are practical barriers to progression. But other such facilities would not: imagine a machine that could search through all the billions of books, recordings and artworks that humanity has created, and surface the undiscovered genius, to give its [[Bayesian prior]]s a fighting chance?
A game of Chinese Whispers among a group of [[Turing machine]]s would make for a grim evening.


''This is not just the Spotify algorithm'', as occasionally delightful as it is. That has its own agenda: revenue maximisation for Spotify is its primary goal, and listener enlightenment is an occasional by-product, where the two intersect. But as long as you are satisfied enough to keep listening, it doesn't care. It is a ''[[cheapest to deliver]]'' strategy: it satisfices. It will tend to serve up populist mush: the reader’s personal “[[cheesecake for the brain]]”. Per {{author|Anita Elberse}}’s {{br|The Blockbuster Effect}}, commercial algorithms, targeted primarily at revenue optimisation, have had the counter-intuitive effect of ''truncating'' the “[[long tail]]” of consumer choice Wired’s Chris Anderson famously envisioned it. A sensible use for this technology would ''extend'' it.  
In any case, you could not assign a human, or any number of humans, the task of “cataloguing the entire canon of human creative output”: this is quite beyond their theoretical, never mind practical, ability. With a machine, at least in concept, you could.<ref>Though the infinite fidelity of machines is overstated, as I discovered when trying to find the etymology of the word “[[satisfice]]”. Its modern usage was coined by Herbert Simon in a paper in 1956, but the Google ngram suggests its usage began to tick up in the late 1940s. On further examination, the records transpire to be optical character recognition errors. So there is a large part of the human oeuvre — the pre-digital bit that has had be digitised— that does suffer from analogue copy errors.</ref>


If artificial intelligence is so spectacular, shouldn’t we be a bit more ambitious in our expectations about what it could do? Isn't “giving you the bare minimum to keep stringing you along” just a little ''underwhelming''?
===== Mind scaling =====
“Scaling”, for imaginative tasks, is different. Here, humans scale well. We don’t want identical, digital, high-fidelity ''duplications'': ten thousand copies of ''Finnegans Wake'' will contribute no more to the human canon than does one.<ref>Or possibly, even ''none'': Wikipedia tells us that, “due to its linguistic experiments, stream of consciousness writing style, literary allusions, free dream associations, and abandonment of narrative conventions, ''Finnegans Wake'' has been agreed to be a work largely unread by the general public.”</ref> Multiple humans doing “mind stuff” contribute precisely that idiosyncrasy, variability and difference in perspective that generates the wisdom, or brutality, of crowds. A community of readers independently parse, analyse, explain, narratise, contextualise, extend, criticise, extrapolate, filter, amend, correct, and improvise the information ''and each others’ reactions to it''.


Imagine a personal large language model, private to the user — free, therefore, of the usual data privacy concerns — that would pattern-match purely by reference to the it's single client’s actual reading habits, prompts and instructions and the recommendations of pattern-matched like-minded readers. This algorithm would be tasked with ''diversifying'' — finding undiscovered works the client might not otherwise find — and rather than ''converging'' — gravitating towards common, popular, commercially promoted ones, it would actively avoid them. .  
This community of expertise is what [[Sam Bankman-Fried]] misses in dismissing Shakespeare’s damning “[[Bayesian prior|Bayesian priors]]”. However handsome William Shakespeare’s personal contribution was to the Shakespeare canon — the text of the actual folio<ref>What counts as “canon” in Shakespeare’s own written work is a matter of debate. There are different, inconsistent editions. Without Shakespeare to tell us, we must decide for ourselves. Even were Shakespeare able to tell us, we could ''still'' decide for ourselves.</ref> — it is finite, small and historic. It was complete by 1616 and hasn’t been changed since. The rest of the body of work we think of as “Shakespeare” is a continually growing body of interpretations, editions, performances, literary criticisms, essays, adaptations and its (and their) infusion into the vernacular. This ''vastly outweighs the bard’s actual folio''.


A rudimentary version of this exists in the LibraryThing recommendation engine, but the scope, with artificial intelligence is huge.
This kind of “communal mind scaling” creates its own intellectual energy and momentum from a small, wondrous seed planted and nurtured four hundred years ago.<ref>On this view Shakespeare is rather like a “prime mover” who created a universe but has left it to develop according to its own devices.</ref> 


This role — seeking out delightful new human endeavours — would be a valuable role ''that is quite beyond the capability of a human'' and which would not devalue, much less usurp the value of human intellectual capacity, but ''empower'' it. This is a suitable application for technology. It respects the division of labour between human and machine.
The wisdom of the crowd thus shapes itself: community consensus has a directed intelligence all of its own. It is not [[utopia|magically benign]], of course, as [[Sam Bankman-Fried]] might tell us, having been on both ends of it.<ref>See also Lindy Chamberlain, Peter Ellis and the sub-postmasters wrongly convicted in the horizon debâcle.</ref>
===Bayesian priors and the canon of ChatGPT===
The “[[Bayesian priors]]” argument which fails for Shakespeare also fails for a [[large language model]].  


Note also the system effect it would have: it would encourage people to create unique and idiosyncratic things. It would distribute wealth along the curve, rather than concentrating it at the top.
Just as most of the intellectual energy needed to render a text into the three-dimensional [[Metaphor|metaphorical]] universe we know as ''King Lear'' comes from the surrounding cultural milieu, so it does with the output of an LLM. The source, after all, is entirely drawn from the human canon. A model trained only on randomly assembled ASCII characters would return only randomly assembled ASCII characters.


We have lying all around us, unused, petabytes of human ingenuity, voluntarily donated into the indifferent maw of the internet. ''We are not lacking ingenuity''. This is one problem homo sapiens ''does not have''. Why would we spend our energy on creating artificial sources of new intelligence?Surely the best way of using this brilliant new generation of machine is to harness the ingenuity that is literally lying around.
But what if the material is not random? What if the model augments its training data with its own output? Might that create an apocalyptic feedback loop, whereby LLMs bootstrap themselves into some kind of hyper-intelligent super-language, beyond mortal cognitive capacity, whence the machines might dominate human discourse?


The JC is not at all bearish on technology in general, or artificial intelligence in particular. He’s just bearish on dopey applications for it.
Are we inadvertently seeding ''Skynet''?


Information technology has done a fabulous job of alleviating boredom, by filling our empty moments with a 5-inch rectangle of gossip, outrage and titillation, but it has done little to nourish the intellect. This is a function of the choices me have made. They, in turn are informed by the interests. Maybe we are missing something by never being bored. Maybe that is a clear space where imagination can run wild. Perhaps being fearful of boredom, by constantly distracting ourselves from our own existential anguish, we make ourselves vulnerable to this two-dimensional online world.
Just look at what happened with [[Alpha Go]]. It didn’t require ''any'' human training data: it learned by playing millions of games against itself. Programmers just fed it the rules, switched it on and, with indecent brevity, it worked everything out and walloped the game’s reigning grandmaster.
 
Could LLMs do that? This fear is not new:{{Quote|{{rice pudding and income tax}}}}
 
But brute-forcing outcomes in fully bounded, [[Zero-sum game|zero-sum]] environments with simple, fixed rules — in the jargon of [[Complexity|complexity theory]], a “tame” environment — is what machines are designed to do. We should not be surprised that they are good at this, nor that humans are bad at it. ''This is exactly where we would expect a Turing machine to excel''.
 
By contrast, LLMs must operate in complex, “[[wicked]]” environments. Here conditions are unbounded, ambiguous, inchoate and impermanent. ''This is where humans excel''. Here, the whole environment, and everything in it, continually changes. The components interact with each other in [[Non-linear interaction|non-linear]] ways. The landscape dances. Imagination here is an advantage: brute force mathematical computation won’t do.{{Quote|Think how hard physics would be if particles could think.
:— Murray Gell-Mann}}
 
An LLM works by compositing a synthetic output from a massive database of pre-existing text. It must pattern-match against well-formed human language. Degrading its training data with its own output will progressively degrade its output. Such “model collapse” is an observed effect.<ref>https://www.techtarget.com/whatis/feature/Model-collapse-explained-How-synthetic-training-data-breaks-AI</ref> LLMs will only work for humans if they’re fed human generated content. [[Alpha Go]] is different.
{{Quote|{{AlphaGo v LLM}}}}
 
There is another contributor to the cultural milieu surrounding any text: the ''reader''. It is the reader, and her “cultural baggage”, who must make head and tail of the text. She alone determines, for her own case, whether it stands or falls. This is true however rich is the cultural milieu that supports the text. We know this because the overture from ''Tristan und Isolde'' can reduce different listeners to tears of joy or boredom. One contrarian can see, in the Camden Cat, a true inheritor of the great blues pioneers, others might see an unremarkable busker.
 
Construing natural language, much less visuals or sound, is no matter of mere [[Symbol processing|symbol-processing]]. Humans are ''not'' [[Turing machine|Turing machines]]. A text only sparks meaning, and becomes art, in the reader’s head. This is just as true of magic — the conjurer’s skill is to misdirect the audience into ''imagining something that isn’t there.'' The audience supplies the magic.
 
The same goes of an LLM — it is simply ''digital'' magic. We imbue what an LLM generates with meaning. ''We are doing the heavy lifting''.
 
===Coda===
{{Quote|{{abgrund}}
:—Nietzsche}}
Man, this got out of control.  
 
So is this just Desperate-Dan, last-stand pattern-matching from an obsolete model, staring forlornly into the abyss? Being told to accept his obsolescence is an occupational hazard for the JC, so no change there.  
 
But if [[This time it’s different|this really is the time that is different]], something about it feels underwhelming. If ''this'' is the hill we die on, we’ve let ourselves down.
 
''Don’t be suckered by parlour tricks.'' Don’t redraw our success criteria to suit the machines.  To reconfigure how we judge each other to make easier for technology to do it at scale is not to be obsolete. It is to surrender.
 
Humans can’t help doing their own sort of pattern-matching. There are common literary tropes where our creations overwhelm us — ''Frankenstein'', ''[[2001: A Space Odyssey]]'', ''Blade Runner'', ''Terminator'', ''Jurassic Park'', ''The Matrix''. They are cautionary tales. They are deep in the cultural weft, and we are inclined to see them everywhere. The actual quotidian progress of technology has a habit of confounding science fiction and being a bit more boring.
 
LLMs will certainly change things, but we’re not fit for battery juice just yet.
 
Buck up, friends: there’s work to do.


===A real challenger bank===
===A real challenger bank===
Line 179: Line 301:
''This is what machine-oriented solutions inevitably do''. Even ones using [[artificial intelligence]]. (''Especially'' ones using [[artificial intelligence]].) Machines are cheap, quick and easy solutions to hard problems. Everyone who takes the same easy solution will end up at the same place — a traffic jam — a local maximum that, as a result, will be systematically driven into the ground by successive, mediocre market entrants seeking to get a piece of the same action.
''This is what machine-oriented solutions inevitably do''. Even ones using [[artificial intelligence]]. (''Especially'' ones using [[artificial intelligence]].) Machines are cheap, quick and easy solutions to hard problems. Everyone who takes the same easy solution will end up at the same place — a traffic jam — a local maximum that, as a result, will be systematically driven into the ground by successive, mediocre market entrants seeking to get a piece of the same action.


''In principle'', humans can make “educated improvisations” in the face of unexpected opportunities in a way that machines can’t. <Ref>Sure: machines can make random improvisations, and after iterating for long enough may arrive at the same local maxima, but undirected evolution is extraordinarily inefficient way to “frig around and find out”.</ref>
''In principle'', humans can make “educated improvisations” in the face of unexpected opportunities in a way that machines can’t. <ref>Sure: machines can make random improvisations, and after iterating for long enough may arrive at the same local maxima, but undirected evolution is extraordinarily inefficient way to “frig around and find out”.</ref>


There is an ineffable, valuable role optimising those machines, adjusting them, steering them, directing them, feeding in your human insight optimising for the environment as it evolves.   
There is an ineffable, valuable role optimising those machines, adjusting them, steering them, directing them, feeding in your human insight optimising for the environment as it evolves.   
Line 187: Line 309:
You will have built carbon-based Turing machines. We already know that humans are bad at being computers. ''That is why we build computers''. But if we raise our children to be automatons they won’t be good at human magic either.
You will have built carbon-based Turing machines. We already know that humans are bad at being computers. ''That is why we build computers''. But if we raise our children to be automatons they won’t be good at human magic either.


Nor, most likely, will the leaders of banking organisations who employ them. These executives will have made it to the top of their respective greasy poles by steadfast demonstration of the qualities to which their organisations aspire. If a bank elevates algorithms over all else, you should expect its chief executive to say things like, “tomorrow, we will have ''robots behaving like people”. This can only be true, or a good thing, ''if you expect your best people to behave like robots''.
Nor, most likely, will the leaders of banking organisations who employ them. These executives will have made it to the top of their respective greasy poles by steadfast demonstration of the qualities to which their organisations aspire. If a bank elevates algorithms over all else, you should expect its chief executive to say things like, “tomorrow, we will have ''robots behaving like people”. This can only be true, or a good thing, ''if you expect your best people to behave like robots''.''


Robotic people do not generally have a rogue streak. They are not loose cannons. They no not call “bullshit”. They do not question their orders. They do not answer back.
Robotic people do not generally have a rogue streak. They are not loose cannons. They no not call “bullshit”. They do not question their orders. They do not answer back.


And so we see: financial services organisations do not value people who do. They value the ''fearful''. They elevate the rule-followers. They distrust “human magic”, which they characterise as human ''weakness''. They find it in the wreckage of [[Enron]], or [[Financial disasters roll of honour|Kerviel]], or [[Madoff]], or [[Archegos]]. Bad apples. [[Operator error]]. They emphasise this human stain over the failure of management that inevitably enabled it. People who did not play by the rules over systems of control that allowed, .or even obliged, them to.
And so we see: financial services organisations do not value people who do. They value the ''fearful''. They elevate the rule-followers. They distrust “human magic”, which they characterise as human ''weakness''. They find it in the wreckage of [[Enron]], or [[Financial disasters roll of honour|Kerviel]], or [[Madoff]], or [[Archegos]]. Bad apples. [[Operator error]]. They emphasise this human stain over the failure of management that inevitably enabled it. People who did not play by the rules over systems of control that allowed, .or even obliged, them to.


The run [[post mortem]]s: with the rear-facing forensic weaponry of [[internal audit]], [[external counsel]] they reconstruct the [[fog of war]] and build a narrative around it. The solution: ''more systems. More control. More elaborate algorithms. More rigid playbooks. The object of the exercise: eliminate the chance of human error. Relocate everything to process.
The run [[post mortem]]s: with the rear-facing forensic weaponry of [[internal audit]], [[external counsel]] they reconstruct the [[fog of war]] and build a narrative around it. The solution: ''more systems. More control. More elaborate algorithms. More rigid playbooks. The object of the exercise: eliminate the chance of human error. Relocate everything to process.''


Yet the accidents keep coming. Our [[financial crashes roll of honour]] refers. They happen with the same frequency, and severity, notwithstanding the additional sedimentary layers of machinery we develop to stop them.  
Yet the accidents keep coming. Our [[financial crashes roll of honour]] refers. They happen with the same frequency, and severity, notwithstanding the additional sedimentary layers of machinery we develop to stop them.  


Hypothesis: these disasters are not prevented by high-modernism. They are a symptom of it. They are its ''products''.
Hypothesis: these disasters are not prevented by high-modernism. They are a symptom of it. They are its ''products''.


===Zero-day vulnerabilities===
===Zero-day vulnerabilities===
Line 211: Line 333:


The age of the machines — our complacent faith in them — has made matters worse. Machines will conspire to ignore “human magic”, when offered, especially when it says “this is not right”.  
The age of the machines — our complacent faith in them — has made matters worse. Machines will conspire to ignore “human magic”, when offered, especially when it says “this is not right”.  
That kind of magic was woven by [[Bethany McLean]]. [[Collateralised debt obligation|Michael Burry]]. [[Harry Markopolos]]. [[Wirecard|Dan McCrum]]. The formalist system systematically ignored them, fired them, tried to put them in prison.
That kind of magic was woven by [[Bethany McLean]]. [[Collateralised debt obligation|Michael Burry]]. [[Harry Markopolos]]. [[Wirecard|Dan McCrum]]. The formalist system systematically ignored them, fired them, tried to put them in prison.


Line 221: Line 344:
Thus, Cryan says and Evidence Lab implies: ''prepare for the coming of the machines''. Automate ''every'' process. Reduce the cost line. Remove people, because when they come for us, ''Amazon won’t be burdened by people''.
Thus, Cryan says and Evidence Lab implies: ''prepare for the coming of the machines''. Automate ''every'' process. Reduce the cost line. Remove people, because when they come for us, ''Amazon won’t be burdened by people''.


===Yes, bank tech is rubbish===
===Yes, bank tech is rubbish ===
To be sure, the tech stacks of most banks ''are'' dismal. Most are sedimented, interdependent concatenations of old mainframes, Unix servers, IBM 386s, and somewhere in the middle of the thicket will be a wang box from 1976 with a CUI interface that can’t be switched off without crashing the entire network. These patchwork systems are a legacy of dozens of mergers and acquisitions and millions of lazy, short-term decisions to patch obsolescent systems with sellotape and glue rather than overhauling and upgrading them properly.  
To be sure, the tech stacks of most banks ''are'' dismal. Most are sedimented, interdependent concatenations of old mainframes, Unix servers, IBM 386s, and somewhere in the middle of the thicket will be a wang box from 1976 with a CUI interface that can’t be switched off without crashing the entire network. These patchwork systems are a legacy of dozens of mergers and acquisitions and millions of lazy, short-term decisions to patch obsolescent systems with sellotape and glue rather than overhauling and upgrading them properly.  


They are over-populated, too, with low-quality staff. Citigroup claims to employ 70,000 technology experts worldwide, and, well, [[Citigroup v Brigade Capital Management|Revlon]].  
They are over-populated, too, with low-quality staff. Citigroup claims to employ 70,000 technology experts worldwide, and, well, [[Citigroup v Brigade Capital Management|Revlon]].  


It ''is'' hard to imagine Amazon accidentally wiring half a billion dollars to customers because of crappy software and playbook- following school leavers in a call centre in Bucharest. (Can we imagine Amazon using call centres in Bucharest? Absolutely). Banks have a first-mover ''dis''advantage here, as most didn’t start thinking of themselves as tech companies until the last twenty years, by which stage their tech infrastructure was intractably shot.  
It ''is'' hard to imagine Amazon accidentally wiring half a billion dollars to customers because of crappy software and playbook- following school leavers in a call centre in Bucharest. (Can we imagine Amazon using call centres in Bucharest? Absolutely). Banks have a first-mover ''dis''advantage here, as most didn’t start thinking of themselves as tech companies until the last twenty years, by which stage their tech infrastructure was intractably shot.  


But the banks have had decades to recover, and if they didn’t then, they definitely do now think of themselves as tech companies. Bits of the banking system — high-frequency trading algorithms, blockchain and data analytics used on global strategies — are as sophisticated as anything Apple can design.<ref>That said, the signal processing capability of Apple’s consumer music software is pretty impressive.</ref>
But the banks have had decades to recover, and if they didn’t then, they definitely do now think of themselves as tech companies. Bits of the banking system — high-frequency trading algorithms, blockchain and data analytics used on global strategies — are as sophisticated as anything Apple can design.<ref>That said, the signal processing capability of Apple’s consumer music software is pretty impressive.</ref>
Line 236: Line 359:
Bank technology is not, of itself, a competitive threat. It is just the ticket to play.
Bank technology is not, of itself, a competitive threat. It is just the ticket to play.


=== Yes, bank staff are rubbish ===
=== Yes, bank staff are rubbish===
Now, to lionise the human spirit ''in the abstract'', as we do, is not to say we should sanctify bank employees as a class ''in the particular''. The JC has spent a quarter century among them. They — we — may be unusually paid, for all the difference we make to the median life on planet Earth, but we are not unusually gifted or intelligent.  
Now, to lionise the human spirit ''in the abstract'', as we do, is not to say we should sanctify bank employees as a class ''in the particular''. The JC has spent a quarter century among them. They — ''we'' — may be unusually well-paid, for all the difference we make to the median life on planet Earth, but we are not unusually gifted or intelligent.  Come on, guys: — [[backtesting]]. [[Debt value adjustments]]. [[ David Viniar|“Six sigma” events several days in a row]]. [[Madoff]].
 
It is an ongoing marvel how commercial organisations can be so reliably profitable given the median calibre of those they employ to steer themselves. Sure, our levels of formal accreditation may be unprecedented, but levels of “[[metis]]” — which can't be had from the academy — have stayed where they are. Market and organisational [[system]]s tend to be configured to ''[[mediocrity drift|ensure]]'' reversion to mediocrity over time.<ref>See {{br|The Peter Principle}}; {{br|Parkinson’s Law}}: for classic studies.</ref>
 
There is some irony here. As western economies shifted from the production of ''things'' to the delivery of ''services'' over the last half-century, the proportion of their workforce in “white collar work” has exploded. There are more people in the UK today than there were in 1970, and almost none of them now work down the mines, on the production line or even in the menswear department at Grace Brothers, as we are given to believe they did a generation ago.  


It is an ongoing marvel how commercial organisations can be so reliably profitable given the calibre of the hordes they employ to steer them. We have argued [[mediocrity drift| elsewhere]] that informal systems tend to be configured to ''ensure'' staff mediocrity over time. Others have too.<ref>See {{br|The Peter Principle}}; {{br|Parkinson’s Law}}: for classic studies.</ref>
All kinds of occupations that scarcely existed when our parents were young have emerged, evolved, and self-declared themselves to be professions. The ratio of jobs requiring, ''de facto'', university degrees (the modern lite professional qualification) has grown. The number of universities have expanded as polytechnics rebadged themselves. This, too, feels like a [[system effect]] of the [[modernist]] orthodoxy: if we can only assess people by reference to formal criteria, then industries devoted to contriving and dishing out those criteria are sure to flourish. The self-interests of different constituencies in the system contrive to entrench each other: this is how [[feedback loop]]s work.


There is some irony here. As western economies have shifted, from the production of ''things'' to the delivery of ''services'', the proportion of  their workforce in “white collar work” has exploded. All kinds of occupations that scarcely existed a generation ago have weaponised themselves into self-declared professions. The ratio of jobs requiring, de facto, university degrees (the modern lite professional qualification) has grown. The number of universities have expanded; polytechnics rebadging themselves. This, too, feels like a [[system effect]] of the modernist orthodoxy: if we can only assess people by reference to formal criteria, then industries devoted to contriving and dishing out those criteria are sure to emerge. The self-interests of different constituencies in the system contrive to entrench each other: this is how feedback loops work
As technologies expand and encroach, taming unbroken landscapes and disrupting already-cultivated ones, let us take a moment to salute the human ingenuity that it takes to defy the march of the machines. For every technology that “solves” a formal pursuit there arises a second order of required white collar oversight. [[ESG]] specialists have evolved to opine on and steward forward our dashboards and marshall criteria for our environmental and social governance strategies — strategies we were until not happy enough to leave in for guidance by Adam Smith’s invisible hand. [[Blockchain as a service]] has emerged to provide a solution to those who believe in the power of disintermediated networks but want someone else to find in and do the heavy-lifting for them. AI compliance officers and prompt engineers steward our relationship with machines supposedly so clever they should not need hand-holding.


Other feedback loops emerge to counteract them. The high modernist programme can measure by any observable criteria, not just formal qualifications. Its favourite is [[cost]].<ref>The great conundrum posed by [[Taiichi Ohno|Ohno-sensei]]’s [[Toyota Production System]] is why prioritise [[cost]] over [[waste]]? Because cost is numerical and can easily be measured by the dullest book-keeper. Knowing what is [[waste]]ful in a process requires analysis and understanding of the system, and so cannot be easily measured. We go eliminate ''cost'' as a lazy proxy. It makes you wonder why executives are paid so well.</ref> The basic proposition is, “Okay, we need a human resources operation. I can't gainsay that. But it need not be housed in London, nor staffed by Oxbridge grads, if Romanian school leavers, supervised by an alumnus from the Plovdiv Technical Institute with a diploma in personnel management, will do.”
Other feedback loops emerge to counteract them. The high modernist programme can measure by any observable criteria, not just formal qualifications. Its favourite is [[cost]].<ref>The great conundrum posed by [[Taiichi Ohno|Ohno-sensei]]’s [[Toyota Production System]] is why prioritise [[cost]] over [[waste]]? Because cost is numerical and can easily be measured by the dullest book-keeper. Knowing what is [[waste]]ful in a process requires analysis and understanding of the system, and so cannot be easily measured. We go eliminate ''cost'' as a lazy proxy. It makes you wonder why executives are paid so well.</ref> The basic proposition is, “Okay, we need a human resources operation. I can't gainsay that. But it need not be housed in London, nor staffed by Oxbridge grads, if Romanian school leavers, supervised by an alumnus from the Plovdiv Technical Institute with a diploma in personnel management, will do.”
Line 273: Line 400:


Just as you shouldn’t confuse products with services, also don’t confuse services with products. There are parts of the client life cycle that feel tedious, high cost, low value things — client onboarding — that are unusually formative of a client’s impression during onboarding. Precisely because they have the potential to be so painful, and they crop up at the start of the relationship. Turning these into client selling points — can you imagine legal docs being a ''marketing'' tool? — can turn this into a service. Treating an opportunity to handhold a new client as a product and not an opportunity to relationship build misses a trick.
Just as you shouldn’t confuse products with services, also don’t confuse services with products. There are parts of the client life cycle that feel tedious, high cost, low value things — client onboarding — that are unusually formative of a client’s impression during onboarding. Precisely because they have the potential to be so painful, and they crop up at the start of the relationship. Turning these into client selling points — can you imagine legal docs being a ''marketing'' tool? — can turn this into a service. Treating an opportunity to handhold a new client as a product and not an opportunity to relationship build misses a trick.
<references />
Retrieved from "https://jollycontrarian.com/index.php?title=Special:MobileDiff/80468...80927"