Natural language processing: Difference between revisions
Amwelladmin (talk | contribs) No edit summary |
Amwelladmin (talk | contribs) No edit summary |
||
| Line 6: | Line 6: | ||
*'''[[Chat bots]]''': An online, chat buddy to whom [[Sales]] can basic legal questions, thereby saving [[Sales]] the aggravation of having to talk to the [[legal eagles]], and [[legal]] the utter [[tedium]] of having to answer the exact same question to the exact [[Sales]] person three or four times daily. | *'''[[Chat bots]]''': An online, chat buddy to whom [[Sales]] can basic legal questions, thereby saving [[Sales]] the aggravation of having to talk to the [[legal eagles]], and [[legal]] the utter [[tedium]] of having to answer the exact same question to the exact [[Sales]] person three or four times daily. | ||
Now reading any text involves judgment, interpretation and negotiation of ambiguity — and bringing to the text the reader’s own understanding of the legal background — while legal language is crafted to avoid ambiguity — {{maxim|there are no metaphors in a trust deed}} — there are still infinite ways of expressing the same idea, and if there is one part of the imagination a lawyer loves to stretch, it is | Now reading any text involves judgment, interpretation and negotiation of ambiguity — and bringing to the text the reader’s own understanding of the legal background — while legal language is crafted to avoid ambiguity — {{maxim|there are no metaphors in a trust deed}} — there are still infinite ways of expressing the same idea, and if there is one part of the imagination a lawyer loves to stretch, it is the part that invents burlesque ways of saying simple things. | ||
In any case, to understand a well-formed English sentence is not just a matter of applying basic rules of language. It is a dynamic process. So expect [[natural language processing]] to be easier said than done. | |||
And so it proves. | And so it proves. | ||
===Legal agreement review=== | ===Legal agreement review=== | ||
:“''[[AI]] can only follow instructions.The [[meatware]] can make a call that the instructions are stupid.''” | :“''[[AI]] can only follow instructions. The [[meatware]] can make a call that the instructions are stupid.''” | ||
There is a well-known and widely feted [[natural language processing]] application<ref>Which shall remain nameless, though you don’t have to be a total ''nerd'' to know who we have in mind.</ref> which purports to save resources and reduce risk by performing a preliminary review of, say, [[confidentiality agreement]]s against a preconfigured [[playbook]]. | There is a well-known and widely-feted [[natural language processing]] application<ref>Which shall remain nameless, though you don’t have to be a total ''nerd'' to know who we have in mind.</ref> which purports to save resources and reduce risk by performing a preliminary review of, say, [[confidentiality agreement]]s against a preconfigured [[playbook]]. | ||
The idea is [[triage]]. The application scans the agreement and, using its [[natural language processing]], will pick up the policy points, compare them with the [[playbook]] and highlight them so the poor benighted lawyer can quickly deal with the points and respond to the negotiation. The software [[vendor]] proudly points to a comparison of their software against human equivalents in picking up policy points in a sample of agreements. The software got 94% of the points. The [[meatware]] only got 67%. The Software was quicker. And — chuckle — it needed less coffee. Headline: ''dumb machine beats skilled human''. | The idea is [[triage]]. The application scans the agreement and, using its [[natural language processing]], will pick up the policy points, compare them with the [[playbook]] and highlight them so the poor benighted lawyer can quickly deal with the points and respond to the negotiation. The software [[vendor]] proudly points to a comparison of their software against human equivalents in picking up policy points in a sample of agreements. The software got 94% of the points. The [[meatware]] only got 67%. The Software was quicker. And — chuckle — it needed less coffee. Headline: ''dumb machine beats skilled human''. | ||
Revision as of 16:19, 2 September 2019
|
The JC pontificates about technology
An occasional series.
|
A great hope of reg tech is natural language processing, which presents itself in a handful of varieties of the same thing: a machine that reads contracts for you.
Examples
- Data extraction: Crawling over your portfolio of 40,000 ISDA Master Agreements[1] to extract the 60 key trading and credit terms out of them that the firm neglected to collect over the last 25 years while it was signing them up;
- Legal agreement review: algorithmically scanning standard-form contracts[2] to identify key terms and risk provisions and save human lawyers from that tedious chore;
- Chat bots: An online, chat buddy to whom Sales can basic legal questions, thereby saving Sales the aggravation of having to talk to the legal eagles, and legal the utter tedium of having to answer the exact same question to the exact Sales person three or four times daily.
Now reading any text involves judgment, interpretation and negotiation of ambiguity — and bringing to the text the reader’s own understanding of the legal background — while legal language is crafted to avoid ambiguity — there are no metaphors in a trust deed — there are still infinite ways of expressing the same idea, and if there is one part of the imagination a lawyer loves to stretch, it is the part that invents burlesque ways of saying simple things.
In any case, to understand a well-formed English sentence is not just a matter of applying basic rules of language. It is a dynamic process. So expect natural language processing to be easier said than done.
And so it proves.
Legal agreement review
There is a well-known and widely-feted natural language processing application[3] which purports to save resources and reduce risk by performing a preliminary review of, say, confidentiality agreements against a preconfigured playbook.
The idea is triage. The application scans the agreement and, using its natural language processing, will pick up the policy points, compare them with the playbook and highlight them so the poor benighted lawyer can quickly deal with the points and respond to the negotiation. The software vendor proudly points to a comparison of their software against human equivalents in picking up policy points in a sample of agreements. The software got 94% of the points. The meatware only got 67%. The Software was quicker. And — chuckle — it needed less coffee. Headline: dumb machine beats skilled human.
But this may highlight a shortfall, not a feature, in the application. The day a palaver of risk controllers set their playbook parameters at their exact hard walkaway point is the day Mr. Gorsky gets to the moon. So, not everything in the playbook says is a problem really is a problem. Much of a playback will be filled with nice-to-haves and other paranoid ramblings of a chicken lickeny somewhere in a controller group. The very value a lawyer brings is to see a point and say, "yeah, that's fine, jog on, nothing to see here. That is the one thing a natural language processing AI can’t do. The AI forces you to negotiate all playbook points, regardless of how stupid they are. True: this isn’t the AI’s fault, but it is inevitable, and it is the AI’s limitation.
AI can only follow instructions.The meatware can make a call that the instructions are stupid.
And besides, having the AI spot the issues and asking the meatware to fix the drafting gets the triage backwards. Picking up the points — and recognising the stupid parts of the playbook is the “high value work”. That is what the meatware should be doing. Fixing the drafting is the dreary detail. That is where you want the AI to kick in. But contextually amending human language — you know, real “natural language processing” — is hard. No AI that we have seen just yet can do it.
And how comfortable can we really be that the AI has spotted everything? If we assume — colour me cynical — the “natural language processing” isn’t quite as sophisticated as its marketers would have you believe[4] then it is a bit reckless to put your faith in the reg tech. Is there no human wordsmith who could fool the AI?[5] what if there is an odious clause not anticipated by the playbook?[6] If the meatware can’t wholly trust the AI to have identified all salient points the lawyer must still read the whole agreement to check. Ergo, no time or cost saving.
Furthermore the reality is that many of the policy points in the playbook will be non-essential "perfect world" recommendations (“nice-to-haves”) which an experienced negotiator will quickly be able to wave through in most circumstances.
But this software is designed to facilitate "right-sourcing" the negotiation to cheaper (ergo less experienced) negotiators who will rely on the playbook as guidance, will not have the experience to make a commercial judgement unaided and will therefore be obliged either to escalate, or to engage on a slew of nice-to-have but bottom-line unnecessary negotiation points with the counterparty. Neither are good outcomes. Again, an example of reg tech creating waste in a process where investment in experienced human personnel would avoid it.
The basic insight here is that if a process is sufficiently low in value that experienced personnel are not justified, it should be fully automated rather than partially automated and populated by inexperienced personnel
References
- ↑ You know, the ones printed on faded waxy fax paper and languishing in filing cabinets around the trading floor; the ones scanned into a 57 MB tiff file along with three amendments, forty pages of specimen signatures, a power of attorney, hand-annotated emails from Credit and the five key pages of the Schedule missing; the ones that are misfiled as Swiss rahmenvertrags; the ones that are just not there at all.
- ↑ To date, only one any one has successfully managed is the one that no-one really cares about: the confidentiality agreement
- ↑ Which shall remain nameless, though you don’t have to be a total nerd to know who we have in mind.
- ↑ That is is a glorified key-word search, in other words.
- ↑ I bet I could. It is hardly challenging to insert an indemnity which does not use the words “indemnity”, “hold harmless” or “reimburse”.
- ↑ Given how fantastically paranoid a gathering of risk controllers can be this seems a remote risk, I grant you, but risks are fractal, remember. And emergent in unexpectable ways. The collective noun for a group of risk controllers is a Palaver, by the way.