Template:M intro work Large Learning Model: Difference between revisions

{{d|LLM|/ɛl ɛl ɛm/ (''also “[[large language model]]”'')|n|}}

Once upon a time, an [[LLM]] was a “Master of Laws”: the postgraduate mark of the ''sensei'' in the society of legal service providers — either of that, or of the indolence of one not prepared to strike out and put what she has learned into practice — but still: it spoke to perseverance, depth, comprehension and mastery, however pigeon-hearted its motivation.

''Who benefits'', primarily, from this emergent technology? Experience should tell us that the first  — and often the ''last'' — to benefit from legal productivity tools are the ''lawyers''. Should we expect [[this time is different|this time be different]]?

And bear in mind it will be the lawyers who deploy [[LLM]]s as a tool, not their clients. Why? ''Because of that [[ineffability]]''. An [[LLM]] is a pattern-matching device. It understands nothing. It cannot provide unmediated legal advice. It can only ever be a “back-breaker”: the “last mile” needs a human who knows what she is doing, understands the context and complicated human psychology at play in the cauldron of 1 [[negotiation]]. An [[LLM]] can draw pretty, impressive-at-a-distance doodles, but it cannot do that. Nor can it write [[legal opinion]]s — well, not meaningful ones — and nor, unmediated by a law firm, does it have the insurance policy or deep, suable pockets for which a client is paying when it seeks legal advice in the first place.

“But, [[JC]], come on. Be realistic. It is dog-eat-dog out there. Any lawyer keeps the bounty of the [[LLM]] from her clients will soon have her lunch eaten by others who won’t. You cannot fight the invisible hand. We are in a race to the bottom.”

Nor should we underestimate the overwhelming power of the lawyer’s intuition that ''what has gone before is sacrosanct''.

Fittingly, the first chatbot was a designed as a parlour trick. In 1966 Joseph Weizenbaum, a computer scientist at MIT created “[[ELIZA]]” to explore communication between humans and machines. [[ELIZA]] used pattern matching and substitution techniques to generate realistic conversations. By today’s standards, [[ELIZA]] was rudimentary, simply regurgitating whatever was typed into it, reformatted as an open-ended statement or question, thereby inviting further input. As a session continued, the user’s answers became more specific and elaborate, allowing [[ELIZA]] to seem ever more perceptive in its responses.  

[[ELIZA]] was a basic “keepy uppy” machine.

Still, it proved surprisingly addictive — even to those who knew how it worked. Weizenbaum was famously shocked how easily people, including  his own secretary, were prepared to believe [[ELIZA]] “understood” them and contributed meaningfully to the interaction.  

This is, of course, how any kind of mind-reading works: with the right kind of clever questions the conjurer extracts from the subject herself all the information needed to create the illusion.  

[[LLM]]s work the same way. Like all good conjuring tricks, [[generative AI]] relies on misdirection: its singular genius is that it lets us misdirect ''ourselves'', into wilfully suspending disbelief, never noticing who is doing the creative heavy lifting needed to turn its output into magic: ''we are''.

Yet again we are suborning ourselves to the machines. We need to kick this habit.

The irony is that we are [[Neuro-linguistic programming|neuro-linguistically programming]] ''ourselves'' to be wowed by LLMs. By writing prompts, we create our own expectation of what we will see, and when the pattern-matching machine produces something roughly like it, we use our own imaginations to frame and filter and sand down rough edges to see what we want to see, to render that output as commensurate as possible to our original instructions.  

We say, “fetch me a tennis racquet”, and the machine comes back with something resembling a lacrosse stick, we are far more impressed than we would be of a human doing the same thing. We would think the human useless. Dim. But with [[generative AI]] we don’t, at first, notice we are not getting what we asked for. We might think, “oh, that will do.” or, perhaps, “try again, but make the basket bigger, the handle shorter, and tighten up the net.”

This is most obvious in AI-generated art, which famously struggles with hands, eyes, and logically possible three-dimensional architecture. First impressions can be stunning, but a closer look reveals an absurdist symphony. Given how large learning models work, this should bit suprise us. They are all trees, no wood.  

It is just as true of text prompts. AI-generated text looks fabulous at first blush; inspect it more closely and the photorealistic resonance [[emerges]] out of logical cul-de-sacs and two-footed hacks. The fact that they so beguile us is because much of what humans write comprises logical cul-de-sacs and two-footed hacks, but that is another story.

But the novelty soon wears off: as we persevere, we begin to see the magician’s wires. There are familiar tropes; we see how the model goes about what it does. It has its tics. It becomes progressively less surprising eventually settling into an entropic quotidian. It loses lustre.

As does the appeal of generating targeted specific work product iteratively using a random word generator. The first couple of passes are great, they go from zero to 0.5, but the marginal improvement in each round diminishes, and the machine reaches asymptotically towards in upper bound of what you had in mind, which is about 75% of it.  

As generative AI evolves that threshold may move towards 100, — there are some indications it may get worse, see below — but it will not get there, and each round becomes increasingly time-consuming and fruitless, human enthusiasm will wane long before the [[singularity]].

Also why when we are targeting a specific outcome, it is frequently frustratingly not quite what we had in mind

We refine and elaborate our query; we learn how to better engineer prompts, and this becomes the skill, rather than the process of pattern matching to respond to it. so ours is the skill going in, and ours is the skill narratising the output. That is true of all literary discourse: just as much of the “world-creation” goes on between the page and the reader as it does between writer and text.

Letting the reader do the imaginative work to fill in the holes is fine when it comes to literature — better than fine, in fact: it characterises the best art. But it is not how ''legal'' discourse works.  

The ''last'' thing a legal drafter wants is to cede control of the narrative to the reader. Rather, a draft seeks to squash exactly the ambiguity that the metaphors of good literature require.

Legal drafting is not literature in any sense: it reduces the reader to a machine: it hard codes unambiguous meaning. It leaves as little room as possible for interpretation. This is why [[legalese]] is so laboured. It is designed to remove all doubt, ambiguity and ''fun'' out of the reading process. It renders it mechanical, precise and reliable. It bestows certainty at the cost of possibly regardless of the cost, and to hell with literary style and elegance.

We should regard legal drafting as closer to computer code colon a form of symbol processing where the meaning resides in and is limited by The format of the text. That is to solve common there is no room for imaginative world creation.

We don’t doubt that LLM is coming, nor that the legal industry will find a use for it: just that there is a ''useful'', sustained use for it. It feels more like a parlour trick: surprising at first, diverting after a while, but then the novelty wears off, and the appeal of persevering with what is basically a gabby but unfocussed child wears pales.

==== Coda: is ChatGPT getting worse? ====

In other news, scientists are concerned that [[ChatGPT]] might be getting worse. Studies indicate that its accuracy at tasks requiring computational accuracy, like playing noughts and crosses or calculating prime numbers, is rapidly diminishing.