Contract analysis

The Devil’s Advocate™ Go Premium Sir! Sir! I've found an indemnity! In which the curmudgeonly old sod puts the world to rights. Index — Click ᐅ to expand: The Devil’s Advocate Comments? Questions? Suggestions? Requests? Insults? We’d love to 📧 hear from you. Sign up for our newsletter.

Luminaries, thought leaders and digital prophets will tell you that machines can now read and annotate contracts, such that yon poor legal eagles are no longer needed and will shortly have no choice but to work as pleasure droids for our transistor-based overlords. Quoth one such digital prophet:

... machines are also increasingly encroaching on tasks that, until now, have required a human ability to think and reason. In the legal sphere, for example, J. P. Morgan has developed a system that reviews commercial loan agreements. It does in a few seconds what would have required, they estimate, about 360,000 hours of human lawyer time.

— Daniel Susskind, A World Without Work

Three hundred and sixty thousand hours of professional work carried out in seconds. Sounds — literally — incredible, doesn’t it? Preternaturally intelligent silicon minds scanning and processing gigabytes of text in an instant, extracting all material quirks and issues, like Zen from Blake’s Seven. Get your coat, legal eagles.

Contract analysis or data extraction?

But remember what is going on here: a bank is reviewing thousands of its own contracts. They will be rendered in overwrought prose no doubt, but — even allowing for template variations and evolution, the contracts will all be basically the same.

If there has been a modicum of legal design applied — okay; quite a big if — customer details and economic variables will already be set out in a schedule. The legal terms, however gruesomely articulated, will be homogeneous.

In any case, “reviewing a hundred thousand commercial loan agreements” really means “processing one hundred thousand tables of structured data”. This, doubtless, would take (and cost) an aeon if assigned to a battalion of legal eagles, and would in the process drive each of the poor buggers up the wall, but to be clear it would require absolutely no judgement, let alone legal acumen. It is a matter of transcription.

Assigning a lawyer to this task would, in any era, have been an act of cruel & unusual punishment, not to mention economic folly. It is not news that difference engines can process structured data better than meatware. This use of Pentium processing power is hardly the proverbial horseman on the lawyer’s ridge.

Contract review tools

More potentially transformative are emerging “artificially intelligent” contract review tools. These promise to take a counterparty’s draft, analyse it against a pre-defined playbook and automatically mark it up.

That is legal work. It is a much harder stunt than reviewing your own standard terms. Now, “natural language processing” means the mechanical processing of natural language, not the natural processing of language. It is a dumb, statistical, pattern-recognising process. It doesn’t understand. The AI proposition is that, to a certain point, it doesn’t need to. It can still play a useful role in the process even though it doesn’t.

So however accurate a machine’s mark-up may be, a human will need to read it for sense. Especially with important agreements.

Confidentiality agreements are common fodder for automated contract review, being low risk, substantially standard but formally variable: there is no limit to the profligacy with which legal eagles can waste words in articulating generic commercial requirements; someone needs to check them, but it’s dullsville work. The AI can chop some wood here.

But even here, technology disappoints. In many ways it has made the problem worse.

The problem to be solved

Confidentiality agreements are fiddly, low-risk agreements that everyone wants to get through as quickly as possible to get to the fruitier deal negotiations. But they have buried risks if you don’t watch them. So, your legal eagles need to be on their game to keep the stupid out. And be assured, counterparties will throw lots of stupid at you. Let’s face facts: the fellow in charge of the NDA templates is going to be a Golgafrinchan, not a Top Gun Lawyer.

But it is a faff — an excruciatingly written NDA might take an hour to review, and return. This is not exactly glamorous work, though once in a while it is strangely satisfying. But by reference to the prism of “notional person-hours” through which a chief operating officers views the world, NDA review costs money.

Automated contract review promises to save some of that cost. It checks against pre-defined playbook of walk-away points, like a triage unit at a military hospital, breaking the back of the most tiresome work, then handing over to you for a quick once-over before it goes out the door.

But it isn’t as simple as that

But it turns out unseen text isn’t as easy to review as all that. Even boring syntactical things like plurals, irregular verbs and parentheticals (we lawyers love parentheticals) are hard to code for. In time the machine will get better, but the universe of possible ways of articulating a single idea remains infinite and, while legal eagles are stunted in many aspects of their creativity, when devising ways in which they may torture prose they are not. Along that axis, they are extemporisational geniuses. Thus, most commercial contract review tools hire paralegals, in low-cost jurisdictions, to check the machine’s output before sending anything back.

This has three consequences:

Firstly, it slows down the output. Instead of getting your mark-up immediately, you get it three-quarters of an hour or more later — this is enough time to become comprehensively distracted by something else and, for that matter, to have reviewed the NDA yourself.

Secondly, it adds to the cost. Now, to be sure, reg tech providers are master rent-seekers, but here, in their bench of paralegal sense-checkers, they have direct out-of-pocket costs. Thus, their application must carry a heavy per document charge. Some charge less than a hundred bucks for a document. Some charge three times that much. But automated review shouldn’t cost anything.

The point was to save an hour of Legal’s time. But that is a nominal cost. It is sunk. Your legal eagle doesn’t work to rule. She doesn’t sit on a production line. She’ll just fit the review in, along with all the other random things that hit her desk in a day. So, an incoming NDA doesn’t generate a specific cost and firing it out to for automated contract review doesn’t save it. Unless you can demonstrate that your contract review tool will make some of your lawyers redundant – good luck with that one – it is not saving you any money. It is costing real money.

Thirdly, it isn’t really “reg tech” anymore. Reg tech that needs a human standing behind it is a form of disguised outsourcing. Okay; COOs are cool with outsourcing. But here you are outsourcing to personnel you can’t see, you didn’t hire and you can’t evaluate, but who must be cheaper than your own personnel, since you are now paying for them, on top of this confounded reg tech and the rent your software as a service provider is extracting on the whole operation. If these people really are better than your people, you should hire the reg tech firm as a recruitment consultant, not to provide crappy AI.

It makes for more work downstream

When setting up a contract review tool there are two sets of legal decisions to be made: administrators must configure the general terms of the playbook in the abstract, and legal eagles must handle incoming issues from a counterparty in the particular.

• The playbook: There is no playbook on the planet which stipulates walk-away points at the actual, real-life, points at which the organisation will really walk away. Playbooks are forged in the abstract, without sight of individual mitigants that might accompany any given project. They are modelled not on the golden mean, but the lowest common denominator. They will have too many rules, all informed by the timid credit officer’s refrain: it can’t hurt to ask.
• The negotiation: it is a great comfort to an inhouse lawyer that she can make decisions, concede technical points, and let de minimis terms go, on the fly. This is what gives her wings. This vouches safe her autonomy, her mastery and her purpose. This is why she shows up for work; why she slogged through interminable lectures about promissory estoppel all those years ago. There is something ineffable, even infinite about this skill: it is impervious to measurement; this rich forensic magisterium lies beyond the censorial gaze of internal audit. In this sunlit realm, we legal eagles can truly fly.

But a playbook buggers up this freedom. It chains our young eagle to the ground. Scarcely can she unfurl her wings, when the contract review tool has marked-up the document to rule, cleaving to every stricture of the playbook, however prudish, tiresome or nonsensical. It is like Horton the goddamn Elephant, doggedly sitting on a nest up a tree.

Now, bearing in mind that the point is to save her precious time, it would be odd — predictable, sure, but odd — for our young attorney to undo those fussy, machine-generated corrections, and so to transgress a cyber-aged version of our old friend, the anal paradox.

But it is worse than that: thanks to this fusspot machine, suddenly the legal eagle’s ineffable judgment is measurable. The bureaucrats can run MIS. They can see what she has done. They can report on it. Our legal eagle can be held to formal account. Her yen for self-preservation — which crushes a yen for autonomy like a bug — will discourage any deviation from the guidelines — “good!” squeaks internal audit — even though the guidelines will contain much that is stupid.

And besides, she might be incentivised to comply, but her counterparty won’t. To the contrary, it will explode in an incandescent rage at the casual pettifoggery with which its sacred covenant is being treated. Our brave eagle can expect to be knee-deep in hostile negotiation, ill-tempered conference calls and, after a time, hard-tack reverse-ferret client relationship management all over commercial terms which, had she never used automated contract review, she would never have raised at all.

Look — it is only an NDA, and it will get sorted out, but remember the original point of the software was to eliminate work, not generate more of the stuff.

There is a role for AI

There is a role for automated contract review: not reviewing NDAs, not supplanting humans, but in giving them a scratchpad. If the AI works without a human backstop, and you throw anything at it, and commit to training it as you go, it might be useful after a while. A machine that can pick out indemnities, governing law, limitations of liability and exclusions — just finding them, even if it doesn’t have much to say about them — would be a great help. It need not be perfect, as long as it is trainable.

This is a less ambitious, more realistic, goal. Lower your short-term expectations, and the long-term return could be huge.

But “dunno: it might work. Let’s suck it and see” tends not to fly as a business proposal. But removing rent-seeking intermediaries and treating information processing as an unglamorous utility and not the indescribable magic our thought leaders like to say it is, is the promise that information technology made. If we hold it to its actual bargain, not our giddy expectations, we might get somewhere.

See also

• Why is reg tech so disappointing?
• A World Without Work