Template:M intro technology rumours of our demise

The report of my death was an exaggeration.

—Mark Twain

In 2017, then-CEO of Deutsche Bank John Cryan thought his employees’ days were numbered. Machines would do for them. Not just back office grunts: everyone. Even, presumably, Cryan himself.

“Today,” he warned, “we have people doing work like robots. Tomorrow, we will have robots behaving like people”.

You can see where he was coming from: what with high-frequency trading algorithms, AI medical diagnosis, Alpha Go, self-driving cars: the machines were coming for us. And this was before GPT-3. It has only got worse since: The machines have taken over our routine tasks; soon they will take the hard stuff, too.

Now.

As long as there has been the lever, wheel or plough, humans have used technology to do tedious, repetitive tasks and to lend power and speed to our frail earthly shells. Humans have done this because it is smart: machines follow instructions better than we do — that’s what means is to be an “automaton”. At things they are good at, machines are quicker, stronger, nimbler, cheaper and less error-prone than humans.

But that’s an important condition: as George Gilder put it:

“The claim of superhuman performance seems rather overwrought to me. Outperforming unaided human beings is what machines are supposed to do. That’s why we build them.”

The division of labour

Nowadays, we must distinguish between traditional, obedient, rule-following machies, and randomly-make-it-up large language models — unthinking, probabilistic, pattern-matching machines. LLMs are the novelty act of 2023, at the top of their hype cycle right now, like blockchain, was a year ago, and like DLT they will struggle to find an enduring use case.

Traditional machines make flawless decisions, as long as both question and answer are pre-configured. Meatsacks are better at handling ambiguity, conflict and novel situations. We’re not flawless — that’s part of the charm — but wherever we find a conundrum we can at least have a bash. We don’t crash. We don’t hang until dialogue boxes close. That’s the boon and the bane of the meatware: you can’t always tell when a human makes a syntax error.

This is how we’ve always used technology: the human figures out which field to plough and when; the horse ploughs it.

While technology may have prompted the odd short-term dislocation — the industrial revolution put a bunch of basket-weavers out of work — the long-term prognosis has been benign: technology has, for millennia, freed us to do things we previously had no time to try.

Technology opens up the design space. It reveals adjacent possibilities, expands the intellectual ecosystem, domesticates what we know and opens up frontiers to what we don’t.

Frontiers are places where we need smart people to figure out new tools and new ways of operating. Machines can’t do it.

But technology also creates space and capacity to indulge ourselves. Parkinson’s law states: work expands to fill the time allotted for its completion. Technology also frees us up to care about things we never used to care about. The microcomputer made generate duplicate and distribute documents far, far easier. There’s that boon and bane, again.

So, before concluding that this time the machines will put us out of work we must explain how. Why is this time different? What has changed?

FAANGs ahoy?

In 2018, then head of the UBS Evidence Lab — a “sell-side team of experts that work across 55+ specialized areas creating insight-ready datasets” — remarked that the incipient competition for banks was not “challenger” banks, but Apple, Amazon, or Google.

The argument was this: banking comes mostly down to three components: technology, reputation, and regulation.

Two of these — technology and reputation — are substantial problems. The other — regulation — is formalistic, especially if you have a decent technology stack.

So, how do the banks stack up against the FAANGS?

Banks v FAANGs: showdown.

	Technology	Reputation	Regulation
Banks	Generally legacy, dated, patched together, under-powered, under-funded, conflicting, liable to fall over, susceptible to hacking.	Everyone hates the Financial Services industry.	All over it. Capitalised, have access to reserve banks, connected, exchange memberships, etc.
FAANGS	Awesome: state of the art, natively functional, at cutting edge, well-funded, well-understood, robust, resilient. Ok could be hacked	Who doesn’t love Amazon? Who wouldn’t love to have an account at the iBank? Imagine if banking worked like Google Maps!	OK there is a bit of investment required here — and regulatory capital is a thing — but nothing is insurmountable with the Amazon Flywheel no?
Winner	Cmon: are you kidding me? FAANGS all the way!	FAANGS. Are banks even on the paddock?	Banks have the edge right now. But look out white-shoe types: The techbros are coming for you.

That the FAANGS will wipe the floor with any bank when it comes to technology is taken res ipsa loquitur — it needs little supporting evidence, just based on how lousy bank tech is — and, sure, the FAANGs have better standing with the public. Who doesn’t love Apple? Who does love Wells Fargo?

Therefore, the argument goes, the only place where banking presently has an edge is in regulatory licences and approvals, capital, and regulatory compliance. It’s wildly complex, fiendishly detailed, the rules differ between jurisdictions, and the perimeter between one jurisdiction and the next is not always obvious. To paraphrase Douglas Adams: “You might think GDPR is complicated, but that’s just peanuts compared to MiFID.”

But, but, but — any number of artificially intelligent startups can manage that regulatory risk, right?

But really. Let’s park a few uncomfortable facts and give Evidence Labs the benefit of the doubt:

So where are they?

Firstly — if banking is such a sitting duck for predator FAANGS, where the hell are they? It is 2023, for crying out loud. Wells Fargo is still with us. None of Apple, Amazon, or Google as so much as cast a wanton glance in its direction of let alone the Vampire Squid’s. Something is keeping the techbros away.

Techbros aren’t natural at banking

And it’s not just fear of regulation, capital and compliance: if it were, you would expect tech firms to be awesome at unregulated financial services.

But — secondly — they’re not.

We’ve been treated to a ten-year, live-fire experiment with how good tech firms will be in unregulated financial services — crypto — from which the banks — “trad fi” if you please — and, notably, the FAANGS have mainly stayed away. It hasn’t gone well.

Credulous cryptobros have found, and promptly fallen down, pretty much every open manhole already known to the dullest money manager — and discovered some whole new ones of their own to fall down too. Helpfully, Molly White has keeping a running score. Crypto, despite its awesome tech and fabulous branding, has been a disaster.

Tech brand-love-ins won’t survive first contact with banking

Thirdly — cool gadget maker that pivots to banking and does it well has as much chance of maintaining millennial brand loyalty as does a toy factory that moves into dentistry.

Those Occupy Wall Street gang? Apple fanboys, the lot of them. At the moment. But it isn’t the way trad fi banks go about banking that tarnishes your brand. It’s banking. No-one likes money-lenders. It is a dull, painful, risky business. Part of the game is doing shitty things to customers when they lose your money. Repossessing the Tesla. Foreclosing on the condo. That isn’t part of the game of selling MP3 players.

The business of banking will trash the brand.

Bank regulation is hard

Fourthly — regulatory compliance is not formalistic, and it is not “the easy bit of banking”. If you could solve it with tech, the banks would have long since done it. They gave certainly tried. (Modern banks, by the way, absolutely are technology businesses, in a way that WeWork and X/Twitter are not). Regulations change, contradict, don't make sense, overlap, are fiddly, illogical, often counterproductive and they are subject to to interpretation by regulators, who are themselves fiddly, illogical and not known for their constructive approach to rule enforcement.

Getting regulations wrong can have bad consequences. Even apparently formalistic things like KYC and client asset protection. Banks already throw armies of bodies and legaltech at this and still they are routinely breaching minimum standards and being fined millions of dollars.

The gorillas in the room

A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.

—Robert Heinlein

But in any case, park all the above, for it is beside the point. For it overlooks the same core banking competence that Mr. Cryan did: quality people, and quality leadership.

We have fallen into some kind of modernist swoon, in which we hold up ourselves up against machines, as if techne is a platonic ideal to which we should aspire.

So we set our children modernist criteria, too from the moment they set foot in the classroom. The education system selects for individuals by reference to how well they obey rules, how reliably, and quickly, they can identify, analyse and resolve known, pre-categorised, “problems”. But these are historical problems with known answers. This is a finite game. This is exactly what machines are best at. Why on earth we should be systematically educating our children to compete with machines at things machines are best at is well beyond this old codger.

If we tell ourselves that “machine-like qualities” are the highest human aspiration, we will naturally find ourselves wanting. We make it easy for the robots to take our jobs. We set ourselves up to fail.

But human qualities are different — humans can improvise, imagine, project in space and time, judge, narratise, analogise, interpret, assess — they can conceptualise Platonic ideals — in a way that algorithms cannot and LLMs can’t do except by pattern matching.

And there is the impish inconstancy, unreliability and unpredictability of the human condition — these make humans different, not inferior, to algorithms. They make us difficult to control and manage by algorithm.

And this is the point. We are not meant to be making it easy for machines to manage and control us. By suppressing our human qualities, we make ourselves more legible, machine readable, triageable, categorisable by algorithm. The economies of scale and process efficiencies this yields accrue to the machines and their owners, not us.

Why surrender before kick-off like that?

On being a machine

“Any sufficiently advanced technology is indistinguishable from magic.”

—Arthur C. Clarke’s third law

We are in a machine age.

We call it that because machines have proven consistently good at doing things humans are too weak, slow, inconstant or easily bored to do well: mechanical things.

But state-of-the-art machines, per Arthur C. Clarke, aren’t magic: it just seems like it, sometimes. They are a two-dimensional, simplified model of human intelligence. A proxy: a modernist simulacrum. They are a shorthand way of mimicking a limited sort of sentience, potentially useful in known environments and constrained circumstances.

Yet we have begun to model ourselves upon machines. The most dystopian part of John Cryan’s opening quote was the first part — “today, we have people doing work like robots” — because it accurately describes a stupid present reality. We have persuaded ourselves that “being machine-like’’ should be our loftiest aim. But if we are in a footrace where what matters is simply strength, speed, consistency, modularity, fungibility and mundanity — humans will surely lose.

But we aren’t in that foot race. Strength, speed, consistency, fungibility and patience are the loftiest aims only where there is no suitable machine.

If you have got a suitable machine, use it: let your people do something more useful.

If you haven’t, build one.

Body and mind as metaphors

We are used to the “Turing machine” as a metaphor for “mind” but, for these reasons, it is a bad metaphor. It is unambitious. It does not do justice to the human mind.

Perhaps we could invert it. We might instead use “body” — in that dishonourably dualist, Cartesian sense — as a metaphor for a Turing machine, and “mind” for natural human intelligence. “Mind” and “body” in this sense, are a practical guiding principle for the division of labour between human and machine: what goes to “body”, give to a machine — motor skills; temperature regulation; the pulmonary system; digestion; aspiration — the conscious mind has no business there. There is little it can add. It only gets in the way. There is compelling evidence that when the conscious mind takes over motor skills, things go to hell.

But leave interpersonal relationships, communication, perception, construction, decision-making in times of uncertainty, imagination and creation to the mind. Leave the machines out of this. They will only bugger it up. Let them report, by all means. Let them assist: triage the “conscious act” to hive off the mechanical tasks on which it depends. Let the machines loose on those mechanical tasks. Let them provide, on request, the information the conscious mind needs to make its build its models and make its plans, but do not let them intermediate that plan.

The challenge is not to automate indiscriminately, but judiciously. To optimise, so humans are set free of tasks they are not good at, and thereby not diverted from their valuable work by formal process better suited to a machine. This can’t really be done by rote.

Here, “machine” carries a wider meaning than “computer”. It encompasses any formalised, preconfigured process. A playbook is a machine. A policy battery. An approval process.

AI overreach

Nor should we be misdirected by the “magic” of sufficiently advanced technology, like artificial intelligence, to look too far ahead.

We take one look at the output of an AI art generator and conclude the highest human intellectual achievements are under siege. However good humans artists may be, they cannot compete with the massively parallel power of LLMs, which can generate billions of images some of which, by accident, will be transcendentally great art.

Not only does reducing art to its “Bayesian priors” like this stunningly miss the point about art, but it suggests those who would deploy artificial intelligence have their priorities dead wrong. There is no shortage of sublime human expression: quite the opposite. The internet is awash with “content”: there is already an order of magnitude more content to consume than our collected ears and eyes have capacity to take in. And, here: have some more.

We don’t need more content. What we do need is dross management and needle-from-haystack extraction. This is stuff machines ought to be really good at. Why don’t we point the machines at that?

Remember the division of labour: machines are good at dreary, fiddly, repetitive stuff. There are plenty of easy, dreary, mechanical applications to which machines might profitably put but to which they have not, and with which we are still burdened: folding washing, clearing up the kitchen and changing nappies. For these mundane but potentially life-changing tasks there is, apparently, no technological resolution in sight.

Okay, some require motor control and interaction with the irreducibly messy real world, so there are practical barriers to progression. And robot lawnmowers and vacuum cleaners suggest a route ahead there. (They are cool!)

But other facilities would not: remembering where you put the car keys, weeding out fake news, managing browser cookies, or this: curating the great corpus of human creation, rather than ripping it off.

Digression: Nietzsche, Blake and the Camden Cat

The Birth of Tragedy sold 625 copies in six years; the three parts of Thus Spoke Zarathustra sold fewer than a hundred copies each. Not until it was too late did his works finally reach a few decisive ears, including Edvard Munch, August Strindberg, and the Danish-Jewish critic Georg Brandes, whose lectures at the University of Copenhagen first introduced Nietzsche’s philosophy to a wider audience.

—The Sufferings of Nietzsche, Los Angeles Review of Books, 2018

When Shakespeare wrote, almost all of Europeans were busy farming, and very few people attended university; few people were even literate — probably as low as about ten million people. By contrast, there are now upwards of a billion literate people in the Western sphere. What are the odds that the greatest writer would have been born in 1564?

—Sam Bankman Fried’s “sophomore college blog”

Here Sam Bankman-Fried, with his loopy remarks about William Shakespeare’s “pretty damning Bayesian priors”, had a point, though not the one he thought. Friedrich Nietzsche died in obscurity, as did William Blake and Emily Dickinson.

For these geniuses, the improbability engine worked its magic, even if not in their lifetimes. But how many undiscovered Nietzsches, Blakes and Dickinsons are there, who never caught the light, and now lie sedimented into unreachably deep strata of the human canon? How many living artists are currently ploughing an under-appreciated furrow, cursing their own immaculate “Bayesian priors”? How many solitary geniuses are out there, right now, stampeding towards an obscurity a large language model might save them from?

(I know of at least one: legendary rockabilly singer Daniel Jeanrenaud, the Camden Cat, who for thirty years has plied his trade with a beat-up acoustic guitar on the Northern Line, and once wrote and recorded one of the great rockabilly singles of all time. Here it is, on SoundCloud.)

Digression over.

If AI is a cheapest-to-deliver strategy you are doing it wrong

Cheapest-to-deliver
/ˈʧiːpɪst tuː dɪˈlɪvə/ (adj.)
Of the range of possible ways of discharging your contractual obligation to the letter, the one that will cost you the least and irritate your customer the most should you choose it.

Imagine we each had private large language models at our personal disposal — free, therefore, of data privacy concerns — that could pattern-match by reference to our individual reading and listening histories, our engineered prompts, our instructions and the recommendations of like-minded readers. Our LLM would search through the entire human oeuvre — the billions of books, plays, films, recordings and artworks, known and not, that already exist, but instead of using that information to generate random mashups, it would return existing works from the canon of as yet undiscovered delight?

This is not just the Spotify recommendation algorithm, as occasionally delightful as that is. Any commercial algorithm has its own primary goal: revenue maximisation. A certain amount of “customer delight” may be a necessary by-product, but only as far as it intersects with that primary commercial goal. As long as customers are just delighted enough to keep listening, the algorithm doesn’t care how delighted they are.

Commercial algorithms need only follow a cheapest to deliver strategy: they “satisfice”. Being targeted primarily at revenue optimisation, they will tend to converge upon what is likely to be popular, because that is easier to find. Rather than scanning the entire depth of human content, skim the top and keep the punters happy enough.

This, by the way, has been the tale of the collaborative internet: despite Chris Anderson’s wishful forecast in 2006 that universal interconnectedness would change economics forever — that suddenly it would be costless to service the long tail of global demand, prompting some kind of explosion in cultural diversity — what happened in practice has been the exact opposite. The overriding imperative of scale has obliterated the subtle appeal of diversity, while sudden, unprecedented global interconnectedness has had the system effect of homogenising demand.

Not only has it remained easier to target the fat head than the thin tail, but the tail itself got thinner.

A cheapest-to-deliver strategy will have the counter-intuitive effect of truncating the “long tail” of consumer choice. As the tail contracts, the commercial imperative to target common denominators gets stronger. This is a highly undesirable feedback loop. It will homogenise us. We will become less diverse. We will become more fragile.

Now if artificial intelligence is so spectacular, shouldn’t we be a bit more ambitious about what it could do for us? Isn’t “giving you the bare minimum you’ll take to keep stringing you along” a bit underwhelming?

Digression: Darwin’s profligate idea

By now most accept the magic of evolution by natural selection. This really is magic: a comprehensive account of the origin of life — indeed, the sum of all organic “creation”, as it were — can be reduced to a mindless, repetitive process that we can state in a short sentence.

In a population of organisms with individual traits whose offspring inherit those traits only with random variations, those having traits most suited to the prevailing environment will best survive and reproduce over time.

The economy of design is this process is staggering. The economy of effort in its execution is not.

Evolution is tremendously wasteful. Not just in how it does adapt, but in how much of the time it does not.

The chain of adaptations from amino acids to Lennon & McCartney may have billions of links in it, but that is a model of parsimony compared with the number of adaptations over that time didn’t go anywhere — that arced off into one of design space’s gazillion dead ends and forlornly fizzled out. For every serendipitous mutation there are millions and millions of duds.

Evolution isn’t directed — that is its very super-power — so it fumbles blindly around, fizzing and sparking, and its power is that vanishingly small proportion of mutations that accidentally do something useful. Evolution is a random, stochastic process. It depends on aeons of time and burns unimaginable resources. Evolution solves problems by brute force.

Even though it came about through an evolutionary process, “natural” mammalian intelligence, whose apogee is homo sapiens, isn’t like that. It is directed. We can narratise and hypothesise, remember, learn, and rule out plainly stupid ideas without having to go through the motions of trying them.

All mammals can do this to a degree; even retrievers. Humans happen to be particularly good at it. This is our great superpower: it took three and a half billion years to get from amino acid to the wheel, but 6,000 years to get from the wheel to the Nvidia RTX 4090 GPU.

Now. Large language models are, like evolution, undirected. They are a brute force method, using colossal amounts of energy and processing power. They work by a stochastic algorithm not dissimilar to evolution by natural selection. They can get better by chomping yet more data, faster, in parallel, with batteries of server farms in air-cooled warehouses full of lightning-fast multi-core graphics processors. But this takes colossal amounts of processing power and energy. This is starting to get expensive and hard. We are bumping up against computational limits as Moore’s law conks out, and environmental consequences as the planet does.

For the longest time, computing power has been the cheap, efficient model. That is ceasing to be true. More silicon is not a zero-cost option. We will start to see the opportunity cost to devoting all these resources to something that, at the moment, creates diverting sophomore mashups we don’t actually need.

The one thing homo sapiens doesn’t need is more free information. Our defining quality seems to be that we can’t shut up.

Why burn all this energy creating premium mediocre content when we have eight billion idle organic CPUs, capable of first class content? When there are millennia of existing content just sitting, there unattended?

LibraryThing as a model

So how about using this technology to better exploit our existing natural intelligence, rather than imitating it? Could we create system effects to extend the long tail?

It isn’t hard to imagine how this might work. A rudimentary version exists in LibraryThing’s recommendation engine. It isn’t even wildly clever — LibraryThing has been around for nearly twenty years and doesn’t even use AI: each user lists, by ISBN, the books in her personal library. She can rate them, review them, and the LibraryThing algorithm will compare each users’s virtual “library” with all the other user libraries on the site and return the most similar user libraries to yours. The attraction of this is not the books you have in common but the ones you don’t.

The effect can be uncanny. My closest match is a user called eggnog2085. We have 62 books in common out of the 4,843 books between us. I can browse eggnog2085’s books, in a virtual library. The ones we have in common are among my favourite books. Many of eggnog2085’s books are new to me, and pretty much every one appeals. It is like wandering around a library designed to appeal specifically to me. You can see a meta-algorithm, that takes the most simpatico libraries and cross-references them for the most common books between them that are not in my library.

LibraryThing generates this kind of delight with a relatively crude traditional algorithm. Imagine how it might perform with AI, trawling all of human creation with more granular information about user habits.

Note how this role — seeking out delightful new human creativity — satisfies our criteria for the division of labour in that it is quite beyond the capability of any group of humans to do it, and it would not devalue, much less usurp, genuine human intellectual capacity. Rather, it would empower it.

Note also the system effect it would have: if humans held out hope that algorithms were committed to exploring the long tail of human creativity, and not shepherding people towards bits monetisable head, this would incentivise humans to create unique and idiosyncratic things.

It would have the system effect of distributing wealth and information — that is, strength, not power — along the curve of human diversity, rather than concentrating it at the top.

We have lying all around us, unused, petabytes of human ingenuity, voluntarily donated into the indifferent maw of the internet. We are not lacking ingenuity. This is one problem homo sapiens does not have. Why would we spend our energy on creating artificial sources of new intelligence? Surely the best way of using this brilliant new generation of machine is to harness the ingenuity that is literally lying around.

The JC is not at all bearish on technology in general, or artificial intelligence in particular. He’s just bearish on dopey applications for it.

Information technology has done a fabulous job of alleviating boredom, by filling our empty moments with a 5-inch rectangle of gossip, outrage and titillation, but it has done little to nourish the intellect. This is a function of the choices me have made. They, in turn are informed by the interests. Maybe we are missing something by never being bored. Maybe that is a clear space where imagination can run wild. Perhaps being fearful of boredom, by constantly distracting ourselves from our own existential anguish, we make ourselves vulnerable to this two-dimensional online world.

Division of labour, redux

About that “division of labour”. When it comes to mechanical tasks, machines — especially Turing machines — scale very well, while humans scale very badly. “Scaling” when we are talking about computational tasks means doing them over and over again, in series or parallel, quickly and accurately. Each operation can be identical; their combined effect astronomical. Of course machines are good at this: this is why we build them. They are digital: they preserve information indefinitely, however many processors we use, with almost no loss of fidelity.

You could try to use networked humans to replicate a Turing machine, but the results would be slow, costly and disappointing and the humans would not enjoy it. Humans are slow and analogue. With each touch they degrade information or augment it, depending on how you feel about it. The signal-to-noise ratio would quickly degrade: this is the premise for the parlour game “Chinese Whispers” — with each repetition the signal degrades, with amusing consequences. A game of Chinese Whispers among a group of Turing machines would make for a grim evening.)

In any case, you could not assign a human, or any number of humans, the task of “cataloguing the entire canon of human creative output”: this is quite beyond their theoretical, never mind practical, ability. With a machine, at least in concept, you could.

No surprise: human minds scale badly at “body stuff”. But when it comes to “mind stuff,” they scale magnificently. Here, what we look for in “scaling” is different. We don’t want identical, digital, high-fidelity duplication: ten thousand copies of Finnegans Wake contribute no more to the human canon than does one. Multiple humans doing “mind stuff” contribute precisely that variability, difference in perspective that generates the wisdom — or brutality — of crowds: a complex community of readers can independently parse, analyse, explain, narratise, extend, criticise, extrapolate, filter, amend, correct, and improvise the information and each others’ reactions to it. The crowd shapes itself: human consensus has a directed intelligence all of its own. This community of expertise is what Sam Bankman-Fried overlooks in his dismissal of Shakespeare’s “Bayesian priors”. However handsome William Shakespeare’s own contribution was to the Shakespeare canon — the actual folio — it is finite, small and historic, in that it was completed in 1603 and hasn’t been changed since. The rest of the body of work we think of as “Shakespeare” comprises interpretations, editions, performances, literary criticisms, essays, adaptations and its (and their) infusion into the vernacular, vastly outstrips the master’s actual folio and continues to grow and evolve to this day. This kind of “mind scaling” creates its own intellectual energy and momentum (it is rather like the theory of God as a “ prime mover”). No matter how fast pattern-matching machines run in parallel, however much brute, replicating horsepower they throw at the task, artificial intelligence in the shape of a large learning model, without human steering, can’t do any of this.

(The “directed intelligence of human consensus” is not magically benign, of course, as Sam Bankman-Fried might be able to tell us, having been on both ends of it).

A real challenger bank

Optimised automation has its place. All other things being equal, an organisation that has optimised its machines will do better, in peacetime and when at war, than one that hasn’t.

An organisation where machines are optimised is one whose people are also optimised: maximally free to work their irreducible, ineffable, magic; hunting out new lands, opening up new frontiers, identifying new threats, forging new alliances — playing the infinite game — while uncomplaining drones in their service till the fields, harvest crops, tend the flock, work the pits, carry the rubble away from the coalface and manage known pitfalls to minimise the chance of human error.

Machines are historical

Machines are dispositionally historical. They look backward, narratising by reference to available data and prior experience, all of which hails from the past. They have no apparatus for navigating any part of the future that is not like the past.

Now, the future is not entirely dissimilar from the past. We should be grateful for that. For the human sense of “continuity” to have an adaptive advantage, great swathes of what it encounters day to day must be the same, or passably similar. But the parts of the future that are most like the past are the uninteresting parts. They are the aspects of our shared experience where people continue to do what they have always done: our routines; our commonplaces. If a million people bought sliced bread yesterday, it is a safe bet that a similar number will tomorrow, and on next Tuesday.

These regularities are the safe, boring, on-piste, already-optimised part of the future. The eighty percent. Here the risks, and therefore the returns, are slimmest. Stampeding for this part of the demand curve is a dumb idea.

Rory Sutherland puts it well in his excellent book Alchemy: The Surprising Power of Ideas that Don’t Make Sense: To converge on the same spot as all your most mediocre competitors, leaving the rest of design-space to the unconventional thinkers, is a rum game.

This is what machine-oriented solutions inevitably do. Even ones using artificial intelligence. (Especially ones using artificial intelligence.) Machines are cheap, quick and easy solutions to hard problems. Everyone who takes the same easy solution will end up at the same place — a traffic jam — a local maximum that, as a result, will be systematically driven into the ground by successive, mediocre market entrants seeking to get a piece of the same action.

In principle, humans can make “educated improvisations” in the face of unexpected opportunities in a way that machines can’t.

There is an ineffable, valuable role optimising those machines, adjusting them, steering them, directing them, feeding in your human insight optimising for the environment as it evolves.

Now, the dilemma. If, over thirty years, you have systematically recruited for those who best display machine-like qualities — if that is what your education system targets, your qualification system credentialises and your recruitment and promotion system rewards — your people won't be very good at weaving magic.

You will have built carbon-based Turing machines. We already know that humans are bad at being computers. That is why we build computers. But if we raise our children to be automatons they won’t be good at human magic either.

Nor, most likely, will the leaders of banking organisations who employ them. These executives will have made it to the top of their respective greasy poles by steadfast demonstration of the qualities to which their organisations aspire. If a bank elevates algorithms over all else, you should expect its chief executive to say things like, “tomorrow, we will have robots behaving like people”. This can only be true, or a good thing, if you expect your best people to behave like robots.

Robotic people do not generally have a rogue streak. They are not loose cannons. They no not call “bullshit”. They do not question their orders. They do not answer back.

And so we see: financial services organisations do not value people who do. They value the fearful. They elevate the rule-followers. They distrust “human magic”, which they characterise as human weakness. They find it in the wreckage of Enron, or Kerviel, or Madoff, or Archegos. Bad apples. Operator error. They emphasise this human stain over the failure of management that inevitably enabled it. People who did not play by the rules over systems of control that allowed, .or even obliged, them to.

The run post mortems: with the rear-facing forensic weaponry of internal audit, external counsel they reconstruct the fog of war and build a narrative around it. The solution: more systems. More control. More elaborate algorithms. More rigid playbooks. The object of the exercise: eliminate the chance of human error. Relocate everything to process.

Yet the accidents keep coming. Our financial crashes roll of honour refers. They happen with the same frequency, and severity, notwithstanding the additional sedimentary layers of machinery we develop to stop them.

Hypothesis: these disasters are not prevented by high-modernism. They are a symptom of it. They are its products.

Zero-day vulnerabilities

“Bad apples” find and exploit zero-day flaws in the modernist system, which is what we should expect bad apples to do. They will seek out the vunerabilities and they will exploit them. They will find them exactly where the modernist machines are not looking: Apparently harmless, sleepy backwaters.

But who the bad apples are depends on who is asking, and when.

After-the-fact-bad-apples: Nick Leeson, Jeff Skilling, Ken Lay, Jerome Kerviel, Kweku Abodoli, Elizabeth Holmes, Arif Naqvid, Charlie Javis, Jo Lo, Bernie Madoff, Sam Bankman-Fried.

None of these were bad apples before the fact. They were Heroes. Chairman of NASDAQ. Visionary innovators.

For a fully taxonomised system, that runs entirely by algorithm, however smart, derived from the scar tissue of the past, is literally blind to zero-day vulnerabilities. Unless mediated by people thinking and viewing the world unconventionally, it will repeatedly fail. And this has been the tale of the financial markets since Hammurabi published his code.

The age of the machines — our complacent faith in them — has made matters worse. Machines will conspire to ignore “human magic”, when offered, especially when it says “this is not right”.

That kind of magic was woven by Bethany McLean. Michael Burry. Harry Markopolos. Dan McCrum. The formalist system systematically ignored them, fired them, tried to put them in prison.

The difference between excellent banks and hopeless ones: the transparent informal networks by which a good institution mysteriously avoids landmines, pitfalls and ambushes, while a poor one walks into every one.

You can’t code for that. The same human expertise the banks need to hold their creaking systems together, to work around their bureaucratic absurdities and still sniff out new business opportunities and take a pragmatic and prudent view of the risk — this is not a bug in the system, but a feature.

This is the view from the executive suite. They measure individuals by floorspace occupied, salary, benefits, pension contributions, revenue generated. Employees who don’t generate legible revenue show up on the map only as a liability. The calculus is obvious: why pay someone to do badly what a machine could do cheaper, quicker, and more reliably for free?

Thus, Cryan says and Evidence Lab implies: prepare for the coming of the machines. Automate every process. Reduce the cost line. Remove people, because when they come for us, Amazon won’t be burdened by people.

Yes, bank tech is rubbish

To be sure, the tech stacks of most banks are dismal. Most are sedimented, interdependent concatenations of old mainframes, Unix servers, IBM 386s, and somewhere in the middle of the thicket will be a wang box from 1976 with a CUI interface that can’t be switched off without crashing the entire network. These patchwork systems are a legacy of dozens of mergers and acquisitions and millions of lazy, short-term decisions to patch obsolescent systems with sellotape and glue rather than overhauling and upgrading them properly.

They are over-populated, too, with low-quality staff. Citigroup claims to employ 70,000 technology experts worldwide, and, well, Revlon.

It is hard to imagine Amazon accidentally wiring half a billion dollars to customers because of crappy software and playbook- following school leavers in a call centre in Bucharest. (Can we imagine Amazon using call centres in Bucharest? Absolutely). Banks have a first-mover disadvantage here, as most didn’t start thinking of themselves as tech companies until the last twenty years, by which stage their tech infrastructure was intractably shot.

But the banks have had decades to recover, and if they didn’t then, they definitely do now think of themselves as tech companies. Bits of the banking system — high-frequency trading algorithms, blockchain and data analytics used on global strategies — are as sophisticated as anything Apple can design.

We presume Apple, Google and Amazon, who always have thought of themselves as tech companies, are naturally better at tech and more disciplined about their infrastructure. But you never know.

In any case, a decent technology platform is a necessary, but not sufficient condition to success in banking. You still need gifted humans to steer it, and human relationships to give it somewhere to steer to. The software won’t steer itself.

Bank technology is not, of itself, a competitive threat. It is just the ticket to play.

Yes, bank staff are rubbish

Now, to lionise the human spirit in the abstract, as we do, is not to say we should sanctify bank employees as a class in the particular. The JC has spent a quarter century among them. They — we — may be unusually paid, for all the difference we make to the median life on planet Earth, but we are not unusually gifted or intelligent.

It is an ongoing marvel how commercial organisations can be so reliably profitable given the calibre of the hordes they employ to steer them. We have argued elsewhere that informal systems tend to be configured to ensure staff mediocrity over time. Others have too.

There is some irony here. As western economies have shifted, from the production of things to the delivery of services, the proportion of their workforce in “white collar work” has exploded. All kinds of occupations that scarcely existed a generation ago have weaponised themselves into self-declared professions. The ratio of jobs requiring, de facto, university degrees (the modern lite professional qualification) has grown. The number of universities have expanded; polytechnics rebadging themselves. This, too, feels like a system effect of the modernist orthodoxy: if we can only assess people by reference to formal criteria, then industries devoted to contriving and dishing out those criteria are sure to emerge. The self-interests of different constituencies in the system contrive to entrench each other: this is how feedback loops work

Other feedback loops emerge to counteract them. The high modernist programme can measure by any observable criteria, not just formal qualifications. Its favourite is cost. The basic proposition is, “Okay, we need a human resources operation. I can't gainsay that. But it need not be housed in London, nor staffed by Oxbridge grads, if Romanian school leavers, supervised by an alumnus from the Plovdiv Technical Institute with a diploma in personnel management, will do.”

It is as if management is not satisfied that we are mediocre enough. For a generation management orthodoxy has been to use machines, networks and our burgeoning digital interconnectedness to downskill the workforce, while the scope of that workforce has only grown. Why pay for expert staff to do drudgery in London when you can have school leavers do it for a quarter the cost in Bucharest?

But are few, expensive, talented, centralised workers really better than many, cheap mediocre ones? Is the management resource dedicated to overseeing this sprawling complex of pencil pushing justified?

This signals a lack of respect for the work — if you are happy having school leavers doing it off a playbook, it can’t be that hard — but, equally, a lack of respect for machines — if it really is just a case of following a playbook (an analogue algorithm, after all), then why not just program a machine to do it and save all that HR cost and organisational overhead?

But, thirdly, why are you suffering drudgery at all? Why aren’t you using the great experience and expertise of your people to eliminate drudgery?

There is a negative feedback loop here: the experts in London are able and incentivised to eliminate drudgery. Able because they understand the product and the market, and know well what matters and what doesn’t. Incentivised because this stuff is boring.

Outsourced school-leavers in Romania are not: by design they don’t understand the process — they are only on the park because of a playbook — and they’re not incentivised to remove drudgery because doing so would puts them out of a job. Recall the agency paradox.

So we construct the incentives inside the organisation to cultivate a will to bureaucracy. Complicatedness is somewhere between a necessary evil and a virtue.

We continue to get away with it because of the scale these businesses run on, and because all the competitors engage the same narrow group of management consultancy firms who are all infected with exactly the same philosophy. And no-one got fired for hiring McKinsey.

Sunlit uplands

If you were setting up a challenger bank today, what would you do?

Imagine setting them free: automating the truly quotidian stuff, re-emphasising away from bureaucracy as the greatest good, and towards relationship management and expertise?

Don’t complicate the operational organisation for the sake of unit cost. Properly account for your infrastructure. This means taking a long-cycle view. The cost to Citigroup of its “saving” through under-investment in technology and outsourcing operations teams to the Philippines included it's loss — of reputation, credibility legal costs and management resources — from the Revlon loan debâcle. A proximate outcome of Credit Suisse’s thinning out and downskilling of its risk management function was — well, a laundry list of avoidable disasters.

Decide whether you are offering a product or a service. Products are unitary, homogeneous, and should — if properly designed — require no after-sale service. Mortgages. Deposit accounts. Investment funds. These you should fully automate — you should know enough about the product and your consumers that all variables should be known and coded for.

If you are offering a service, make sure it is a service, and not just a poorly-designed product. A service is a relationship between valuable humans. You can’t outsource a relationship to a low-cost jurisdiction without degrading your service. The cost of excellent client management is not wasted in a service. That is the service.

Just as you shouldn’t confuse products with services, also don’t confuse services with products. There are parts of the client life cycle that feel tedious, high cost, low value things — client onboarding — that are unusually formative of a client’s impression during onboarding. Precisely because they have the potential to be so painful, and they crop up at the start of the relationship. Turning these into client selling points — can you imagine legal docs being a marketing tool? — can turn this into a service. Treating an opportunity to handhold a new client as a product and not an opportunity to relationship build misses a trick.