Template:M intro work Large Learning Model

LLM
/ɛl ɛl ɛm/ (also “large language model”) (n.)
Once upon a time, an LLM was a “Master of Laws”: the postgraduate mark of the sensei in the society of legal service providers — either of that, or of the indolence of one not prepared to strike out and put what she has learned into practice — but still: it spoke to perseverance, depth, comprehension and mastery, however pigeon-hearted its motivation.

If the thoughtleaderati are to be believed, now all one needs for that kind of expertise is a different kind of “LLM”: a “large language model”. Artificial intelligence rendered by a pattern-recognising, parallel-processing chatbot.

The legal profession is to ChatGPT, we hear, as poor old Chrissie Watkins was to Jaws.

But there have been contumelious prophecies of its demise before. In the manner of a blindfolded dartsman, Professor Richard Susskind OBE has been tossing them around for decades. Just by random chance, you would expect one to hit the wall at some point.

Is this big law’s Waterloo? Will ChatGPT do for our learned friends what the meteor did to the dinosaurs?

Or will the lawyers, like cockroaches, survive? Might they even turn this to their advantage?

Cui bono?

Who benefits, primarily, from this emergent technology? Experience should tell us that the first — and often the last — to benefit from legal productivity tools are the lawyers. Should we expect this time be different?

Now, it is a truism that she who has a tool uses it, firstly, to improve her own lot. A commercial lawyer’s “lot” is predicated on two things: (1) time taken, and (2) ineffability: the sense that what she does “passeth all muggle understanding”.

It is a happy accident that, generally, (2) begets (1): the more ineffable something is, the longer it takes, and the harder it is to work with. The longer it takes, the more you can charge.

Commercial legal contracts are like that. Long, and once they have calcified into templates, fiddly. For lawyers, this is a capital state of affairs. It is why no commercial law firm on the planet really cares for plain English. Oh, they all say, they do, of course — but come on.

This is, in itself, a neat “simplification defeat device”: if you make a contract template sufficiently convoluted, the one-off cost of simplifying it so vastly outweighs the cost of just “tweaking” and living with it that few clients will ever take that first step to simplify. Even though the the ongoing costs of not rationalising dwarf the one-off costs of doing so, the long-term savings are always over that hump.

And bear in mind it will be the lawyers who deploy LLMs as a tool, not their clients. Why? Because of that ineffability. An LLM is a pattern-matching device. It understands nothing. It cannot provide unmediated legal advice. It can only ever be a “back-breaker”: the “last mile” needs a human who knows what she is doing, understands the context and complicated human psychology at play in the cauldron of 1 negotiation. An LLM can draw pretty, impressive-at-a-distance doodles, but it cannot do that. Nor can it write legal opinions — well, not meaningful ones — and nor, unmediated by a law firm, does it have the insurance policy or deep, suable pockets for which a client is paying when it seeks legal advice in the first place.

An LLM can only be deployed, that is to say, by someone with skin in the game; who is prepared to put herself in jeopardy by accepting the assignment, which jeopardy she defends by the simple expedient of knowing what she is doing and checking her LLM’s output.

That someone will be a lawyer.

Now such a “last mile” lawyer could use an LLM to simplify documents, accelerate research and break legal problems down to their essences, thereby reducing the cost, and increasing the value, of her service to her clients. And, sure: in theory, she could give all this value up to her clients for nothing.

But she could, just as easily, use an LLM to further complicate the “work product”: to overengineer, to convolute, to invent options and cover contingencies of minimal utility: she could set her tireless symbol-processing engine to the task of injecting infinitesimal detail: she could amp-up the ineffability to a level beyond a normal human’s patience.

Which of these, realistically, should we expect a lawyer to do? Simplify, or complicate? Sacrifice time and ineffability, for the better comprehension of the unspecialised world? Or plough the energy this magical new tool bestows into generating more convolution and ineffability, racking up more recorded time, and building up the bulwark against the muggles?

She would do the latter with only the best intentions, of course; this is not lily-gilding so much as a noble outreach toward perfection: using the arsenal at her disposal to reach ever closer to the Platonic form.

Cynical, or just realistic? Foretellers of legal Armageddon must explain away some difficult facts: that the commercial-legal industrial complex has stubbornly resisted all attempts at simplification and disintermediation for a generation, notwithstanding the thought-leadership, regulatory prompting, appeals to logic and 40 years of enabling technology — Microsoft Word, mainly — which the world’s lawyers could have used, powerfully, to simplify and minimise the legal work product.

Not only did they not do that, they used their tools to make everything more complicated. Boilerplate blossomed. Templates flowered. Even trivial contracts acquired wording dealing with counterparts, governing the form of amendments and excluding third party rights that weren’t there in the first place.

This is a perfect job for ChatGPT. Why should a difference engine designed to generate plausible-sounding but meaningless text be used do anything different?

You can see the effect is is having on legal work product. NDAs grow ever longer, increasingly riven with the same generic ornamentations that usually range between harmless and misconceived but which are now so prevalent — they recur as the LLMs hone their model — as to become hard for the meatware to resist.

The meatware, remember, has limited patience with NDAs, understanding in a way an algorithm cannot how much of a pantomime they are. Algorithms, on the other hand have unlimited patience and boundless energy. If negotiation comes down to who passes out first, we should bear in mind that LLMs don’t pass out.

Who’s client? Oh, right: she’s a lawyer, too.

“But, JC, come on. Be realistic. It is dog-eat-dog out there. Any lawyer keeps the bounty of the LLM from her clients will soon have her lunch eaten by others who won’t. You cannot fight the invisible hand. We are in a race to the bottom.”

But are we?

Ignoring how impervious to the invisible hand all other recent technologies have been, remember who the clients are. Consumers of high-end commercial legal services are not, generally, the permanently bamboozled muggles of common myth. Most are themselves lawyers, inhabiting weaponised legal departments mainly comprised of veteran deal lawyers. These are people also take pride in their ability to work with difficult, complicated things. This is how they prove their worth to their employers.

Lawyer and their clients, that is to say, have a common interest in convolution for its own sake. They are the jazz aficionados of text; cinéastes of syntax. They expect overwrought contracts: nothing says “prudent management of existential risk” like eighty page of 10pt Times New Roman.

Plain English is not for serious people.

Conservative motivation

Nor should we underestimate the overwhelming power of the lawyer’s intuition that what has gone before is sacrosanct.

Lawyers are the last great positivists: they understand instinctively that what has been already laid down by someone else — “posited” — is safer and than anything new that they might themselves contribute. The common law with its doctrine of precedent, after all, is to all intents a divine commandment: in times of doubt, to do what has been done before.

The more authoritative the source, the more sacred it will be.

Thus, lawyers will assiduously “track the wording of legislation” to ensure their drafting matches it with utmost fidelity, notwithstanding any private reservations they may have about how it was drafted. The more ambiguous, or just difficult the source text, the more assiduously should we expect lawyers to replicate it, because they fear it. They fear the limits of their own mastery.

This “positivism-through-fear” extends with equal force to established market precedents. It doesn’t matter how manifestly unfit for purpose it is, the resistance to change will be strong.

Literary theory, legal construction and LLMs

Fittingly, the first chatbot was a designed as a parlour trick. In 1966 Joseph Weizenbaum, a computer scientist at MIT created “ELIZA” to explore communication between humans and machines. ELIZA used pattern matching and substitution techniques to generate realistic conversations. By today’s standards, ELIZA was rudimentary, simply regurgitating whatever was typed into it, reformatted as an open-ended statement or question, thereby inviting further input. As a session continued, the user’s answers became more specific and elaborate, allowing ELIZA to seem ever more perceptive in its responses.

Even though ELIZA was a basic “keepy uppy” machine, it proved surprisingly addictive — even to those who knew how it worked. Weizenbaum was famously shocked how easily people, including his own secretary, were prepared to believe ELIZA “understood” them and contributed meaningfully to the interaction.

This is, of course, how all “mind-reading” works: with the right kind of clever questions, the conjurer extracts from the subject herself all the information she needs to create the illusion.

LLMs work the same way. Like all good conjuring tricks, generative AI relies on misdirection: its singular genius is that it lets us misdirect ourselves, into wilfully suspending disbelief, never noticing who is doing the creative heavy lifting needed to turn machine-made screed into magic: we are. We are neuro-linguistically programming ourselves to be wowed by LLMs.

Yet again, we subordinate ourselves to suit the convenience of the machines. We really need to have some self-respect and kick this habit.

By writing prompts, we create our own expectation of what we will see. When the pattern-matching machine produces something roughly like it, we then use our own imaginations to backfill, frame, filter, correct, boost, sharpen and polish the output into what we want to see. We render that output as commensurate we can with our original instructions.

When we say, “fetch me a tennis racquet”, and the machine comes back with something more like a lacrosse stick, we are far more impressed than we would be had a human done the same thing. We would think the human a bit dim. But with generative AI we don’t, at first, even notice we are not getting what we asked for. We might think, “oh, that will do,” or perhaps, “ok, computer: try again, but make the basket bigger, the handle shorter, and tighten up the net.” We can iterate this way until we have what we want — or we could just use a conventional photo of a tennis racquet.

AI-generated image generation famously struggles with hands, eyes and logical three-dimensional architecture. First impressions can be stunning, but the second look reveals an absurdist symphony. It is just as true of text prompts: on close inspection we can see the countless minute logical cul-de-sacs and two-footed hacks from which the clever story miraculously emerges. (To be sure, many human authors write in logical cul-de-sacs and two-footed hacks, but that is another story.)

Either way, the novelty soon palls and, as we persevere, we begin to see the magician’s wires. We get a sense of how the model goes about what it does: its familiar tropes and tics and persistent ways of doing things which are never quite what you have in mind. The piquant surprise at what it produces dampens at each go-round, eventually settling into an entropic and vaguely dissatisfying quotidian.

In this way, the appeal of iterating a targeted work product with a random pattern-matcher soon loses its lustre. The first couple of passes are great: they get from zero to 0.5. But the marginal improvement in each following round diminishes, as the machine reaches asymptotically towards its upper capability in producing what you had in mind, which we estimate unscientifically as about 75% of it.

Now, as generative AI improves towards 100 — assuming it does improve: there are some indications it may not; see below — that threshold may move but it will never get to 100. In the mean time, as each successive round takes more time and bears less fruit, mortal enthusiasm and patience with the LLM will have long-since waned: well before the Singularity arrives.

And many improvements we will see will largely be in the meatware: as we refine and elaborate our queries; we learn how better to frame our queries, and “prompt-engineering” becomes the skill, rather than the dumb, parallel pattern-matching process that responds to it. Ours is the skill going in, and ours is the skill construing the output. What the machine does is the boring bit.

In all kinds of literature bar one, construal is where the real magic happens: it is the emergent creative act and community consensus that renders King Lear a timeless cultural leviathan and Dracula: The Undead forgettable pap. A literary work may start with the text, but it barely stays there for a moment. The “meaning” of literature is necessarily personal to the reader: it lives between our ears, and within the cultural milieu that interconnects the reading population.

“Construal” and “construction” are interchangeable in this sense: over time that cultural milieu takes the received corpus of literature and, literally, constructs it into edifices its authors can have scarce have imagined. Hamlet speaks, still, to the social and human dilemmas of the twenty-first century in ways Shakespeare cannot possibly have contemplated. to be clear: the reader of literature is no symbol processor, decrypting text to reveal a one-to-one scale assembly of the content of the author’s head. Literature is not an instruction manual.

Now there is one kind of “literature” where the last thing the writer wants is for the reader use her imagination to fill in holes in the meaning. Where clarity of authorial intention is paramount; where communicating and understanding purpose is the sole priority: legal literature.

The last thing a legal drafter wants is to cede interpretative control to the reader. Rather, she seeks to squash all opportunities for improvisation presented by creative ambiguity. This is why, just as there are no atheists in foxholes, there are no metaphors in a Trust Deed.

Legal drafting seeks to be as finite as it can be. It strives do to readers what code does to hardware: to reduce them to mere symbol-processing machines, extracting the author’s single incontrovertible meaning. To leaves as little room as possible for interpretation. That there is such a living to be made conducting commercial litigation demonstrates how hard this is.

This is one reason why legalese is so laboured. It is designed to chase down all blind alleys, previsualise all phantoms and prescribe outcomes for all logical possibilities. To do so, it removes all ambiguity and renders the text as mechanical, precise and reliable as it can be. There are no metaphors in a trust deed. Where normal literature favours possibility over certainty, legal language bestows certainty at the cost of possibility, and to hell with literary style and elegance.

Legal language is, in James Carse’s sense, finite. Literature is infinite.

Now: the punchline. Given how integral the reader and her cultural baggage are to the creative act in normal literature, we can see how, in that domain, a large learning model, which spits out text ripe with possibilities, begging for someone to “construct” it, is a feasible model: to move from a model where most of the creative work is done by the reader to one where all of it is, is no great step.

There is enough bad human literature like that out there now, that is is no great stretch to imagine doing without the human altogether. In that case, what does it matter what the text says, as long as it is coherent enough for an enterprising reader to make something out of it?

But that does not work at all at all for legal language. Legal language is code: it must say exactly what the parties require: nothing more or less, and it must do it in a way that leaves nothing open to a later creative act of interpretation. We should regard legal drafting as closer to computer code than literature: a form of symbol processing where the meaning resides wholly within and is fully limited by the text.

But unlike computer code, the operating system it is written for is not a closed logical system, and even the best-laid code can still run amok. You can’t run it in a sandbox to see if it works.

Meet the new boss —

We don’t doubt that LLM is coming, nor that the legal industry will find a use for it: just that there is a useful, sustained use for it. It feels more like a parlour trick: surprising at first, diverting after a while, but then the novelty wears off, and the appeal of persevering with what is basically a gabby but unfocussed child wears pales.

The traditional legal model faces existential challenges for sure, but they are not presented, and will not be addressed by random word generators.

Coda: is ChatGPT getting worse?

In other news, scientists are concerned that ChatGPT might be getting worse. Studies indicate that its accuracy at tasks requiring computational accuracy, like playing noughts and crosses or calculating prime numbers, is rapidly diminishing.

Perhaps ChatGPT is getting bored, or might it have something to do with the corpus increasingly comprising nonsense text generated on the hoof by some random using ChatGPT?