Template:M intro technology Better call ChatGPT: Difference between revisions

{{drop|A|n interesting finding,}} noted but not explored by the paper, was a variance measurement<ref>“Cronbach’s alpha” is a statistic that measures internal consistency and reliability, of a different items such as, in this case, the legal agreement reviews. A high “alpha” indicates consistency and general agreement; a low alpha indicates variance or disagreement.</ref> across the categories of human reviewers: the ''least'' qualified, the [[LPO]]s had an “alpha” variance of 1.0, implying complete agreement among them about the issues (a function, we suppose, of slavish and obedient adherence that is beaten into LPO businesses). This ''dropped'' to 0.77 for junior lawyers and further still to 0.71 for senior lawyers.

You read that right: experienced lawyers were ''least'' likely to agree what was important in a basic contract.  

This says one of two things: either lawyers get worse at reading contracts as they get more experienced — by no means out of the question, and would explain a few things — or there is something not measured in these [[key performance indicator]]s that sets the veterans apart. That, maybe, linear contract analytics is the proverbial a [[machine for judging poetry]], and isn’t all there is to it.  

But it shouldn’t surprise anyone that all the machines were quicker that the humans of whom. LPOs were by far the slowest. There is a cost to obliging humans to behave like robots.

Clear implication: as we can expect [[LLM]]s to get better over time,<ref>Maybe {{Plainlink|https://www.theregister.com/2023/07/20/gpt4_chatgpt_performance/|not, actually}}, but okay.</ref> the [[meatware]]’s days are numbered.  

There being no “[[bright line test|bright lines]]” wherever there is scope for nuance or call for subtlety, she will stay well inside it, not trusting a dolt — whether naturally or generatively intelligent — to get it right.  

This is common sense and little more than prudent  [[triage]]: well before any practical danger, her amanuenses must report to matron for further instruction. She can then send the machine back into the fray with contextualised instructions, or just handle anything properly tricky herself. This is all good best-practice outsourcing, straight from the McKinsey [[playbook]].

Now, a standard-form contract without at least one howling error is unknown to legal science, so she should expect an assiduous machine reader, so instructed, to be tireless, and quickly tiresome, in rooting out formal discrepancies and bringing them to her attention.  

{{drop|C|ontrary to [[modernist]]}} wisdom — viz., ''thou shalt not rest until all problems are solved'' — descending the fractal tunnel of error is, sometimes, a bad idea. ''Usually'', in fact. Down it are [[snafu]]<nowiki/>s and boo-boos that an experienced lawyer will roll her eyes at, take a moment to sanctimoniously tut about, ''but then let go''.  

But, as Radiant’s [[Sign Here|Alex Hamilton]] remarked to me, the converse is also true. An LLM may pick up and flag for routine non-compliance a clause which, in the context, should sound a deeper alarm. A hedge fund customer who cuts back or even wordsmiths a standard [[market abuse]] representation ''[i.e., that it will never place an order when in possession of non-public price-sensitive information]'' because that ''is'' a [[bright line]] and a willingness even to approach it should be setting off klaxons across the organisation.

A firm must, therefore, have a written contract to do business. To meet that end, ''any'' delay in finalising that contract, in the name of “getting it right”, ought to be a source of regret. (You might be surprised how often financial firms are obliged to negotiate their [[terms of business]], in that it is not “never”.)

{{drop|T|hose exceptions aside}}, where high-volume, low-risk legal processes do not function as courting rituals, the name of the game is not ''perfect'' negotiation, but ''no'' negotiation. This is a crucial distinction: negotiation does not fix the problem: it ''is'' the problem.  

{{Drop|B|y design, LLMs}} learn and reason exclusively from what has gone before. While, yes, [[Alpha Go|AlphaGo]] might have engineered a novel strategy in a [[zero-sum game]], the non-linear infinitude of life that a contract review process is a different kettle of fish. And this is not, in any case, what one wants in a sorcerer’s apprentice.

That being the case the perfect LLM would be one that served up an archetypal sample of ''what you already have''.