Contract analysis

The Devil’s Advocate™ Go Premium Sir! Sir! I've found an indemnity! In which the curmudgeonly old sod puts the world to rights. Index — Click ᐅ to expand: The Devil’s Advocate Comments? Questions? Suggestions? Requests? Insults? We’d love to 📧 hear from you. Sign up for our newsletter.

Luminaries, thought leaders and digital prophets will tell you that machines can now read and annotate contracts, such that yon poor legal eagles are no longer needed and will shortly hacve no choice but to work as pleasure droids for our transistor-based overlords. Quoth one such digital prophet:

... machines are also increasingly encroaching on tasks that, until now, have required a human ability to think and reason. In the legal sphere, for example, J. P. Morgan has developed a system that reviews commercial loan agreements. It does in a few seconds what would have required, they estimate, about 360,000 hours of human lawyer time.

— Daniel Susskind, A World Without Work

Three hundred and sixty thousand hours of professional work carried out in seconds. Sounds — literally — incredible, doesn’t it? Preternaturally intelligent silicon minds scanning and processing gigabytes of diverse text in an instant and analysing it for all material quirks and issues, like Zen from Blake’s Seven. Get your coat, legal eagles.

Contract analysis or data extraction?

But let’s just stop and consider what is actually going on here: a bank seeking to reviewing tens and probably hundreds of thousands of its own commercial loans. These loans will be rendered in the hideous, overwrought prose of the banking lawyer, to be sure, but — even allowing for template variations and evolution (by which I mean periodic Cambrian explosions of flannelry to which not even the House of Morgan is immune), the contracts will all be basically the same. If there has been a modicum of legal design applied — okay, that’s quite a big if — there will be a predictable schedule of customer details and economic variables — dates, amounts, currencies, rates, and optional elections — but the legal terms, however gruesomely articulated, will be homogeneous and, in any case within the bank’s control: a commercial bank with half a brain won’t allow its customers to wordsmith its standard terms. If it does, then that is the problem to be solved, not “how can I possibly understand all these varied terms I have agreed with hundreds of thousands of customers?”

Note the irony: it was only thanks to this kind of technology that we can even contemplate micro-variations to standard terms: behold the technology paradox: what Andy Grove giveth, the legal eagle’s innate appetite for pedantry taketh away. Technology that allows one to ignore the root cause of the problem and carry on anyway is bad technology.

But we digress. In any case, “reviewing one hundred thousand commercial loan agreements” really means “processing one hundred thousand tables of structured data”. This, doubtless, would take (and cost) an aeon if assigned to a battalion of legal eagles, and would in the process drive each of the poor buggers up the wall, but to be clear it would require absolutely no judgement, let alone legal acumen. It is a matter of transcription.

Assigning a lawyer to this task would, in any era, have been an act of cruel & unusual punishment, not to mention economic folly. It is not news that difference engines can process structured data better than meatware. This use of Pentium processing power is hardly the proverbial horseman on the lawyer’s ridge.

Contract review tools

More potentially transformative are emerging “artificially intelligent” contract review tools. These promise to take a foreign draft, received from a counterparty, and analyse it against a pre-defined playbook, together with other examples which the machine has already learned from, mark it up, and return it for completion of the negotiation.

This is a much harder stunt than reviewing your own standard terms. Legal text from random strangers can, and often does, say anything, and is liable to be rendered in the sort of inventively tumid syntax that only a trained professional could confect.

'Semantically, human languages are fundamentally ambiguous. Machines do not understand them. That's not how the AI works. “Natural language processing” means the mechanical processing of natural language, not the natural processing of language. It is a dumb, statistical, pattern-recognising process.

So however accurate a mark-up may become, there will always be a need for a human to read and approve it. A common subject for these contract review tools is confidentiality agreements, being as they are (a) low risk; (b) substantially standard: all confis say the same basic things; and (c) formally variable: there is no limit to the profligacy with which legal eagles can waste words in articulating generic commercial requirements.

But even here, technology disappoints. By and large it has made the existing problem worse. Institutions are now decommissioning contract review systems the same way they are repatriating once offshored operations functions. It is worth investigating why.

The problem statement

What is the problem the contract review tool is designed to solve: Confidentiality agreements are fiddly, but low-risk agreements that one must get through as quickly as possible to get to the fruitier deal negotiations they promise. Confis tend to be an “I’ll show you mind if you show me yours” kind of affair. But — especially in this data-obsessed world — they have buried risks if you don’t watch them carefully.

So you need your legal eagles to be on their game to stop the stupid getting in. And be assured, counterparties will throw lots of stupid at you. Let’s face facts: the fellow in charge of the NDA templates is going to be a Golgafrinchan, not the organisation’s Top Gun Lawyer.

So, confidentiality agreements come in all shapes, sizes and shades of silliness. They needn’t be longer than a few paragraphs, but our American friends are given to 15-page bunker busters which, in the main, say no more than a concise one, but you just never know. Hence, the need for contract review. It is all standard stuff. But it is a faff — an excruciatingly written NDA might take an hour to review, and return. This is not exactly glamorous work, though once in a while it is strangely satisfying.

So: it’s slow, it takes time, the review parameters are complicated (but not complex) — there is a (limited) risk of something slipping in — and a legal eagle has invariably got better things to do. By reference to the illusory “notional person-hours”, through which prism the chief operating officers like to view the world, NDA review costs the firm money.

The contract review tool as a solution.

The contract review tool promises to save some of that money. It performs a basic check against pre-defined playbook of walk-away points. It operates rather like a triage unit at a military hospital, breaking the back of most tiresome work (while you are getting on with something more productive), then handing over to you for a quick once-over and it goes out the door. Brilliant.

That’s the theory.

But it isn’t as simple as that

But it turns out text lovingly confected by some unknown legal eagle isn’t as easy to review as all that. Even boring syntactical things like plurals, irregular verbs and parentheticals (we lawyers love parentheticals) are hard to code for. In time the machine will get better, but the universe of possible ways of articulating a single idea remains infinite and, while legal eagles suffer a form of locked-in syndrome with regard to many aspects of their creativity, it does not extend to methods by which they may torture prose. When it comes to textual complexity and saying things which, for the avoidance of doubt, are not in doubt, lawyers are extemporisational geniuses.

Parsing the fevered language of a human lawyer requires some kind of a human sense check. You would think that could be applied by the lawyer operating the tool but, most commercial contract review tools hire paralegals, in low-cost jurisdictions, to check the output before sending it back.

This has three consequences:

It’s slow

Firstly, it slows down the output: Instead of getting your markup immediately, you get it three-quarters of an hour or more later — that same three-quarters of an hour in which you could have read the NDA yourself. This is more than enough time to become comprehensively distracted by something else.

It’s more expensive

Secondly, it adds to the cost. Now, to be sure, reg tech providers are master rent-seekers, but here, in their bench of paralegal sense-checkers, they have actual out-of-pocket costs. Thus, the contract review tool must carry a heavy charge per document review. Better ones charge less than a hundred bucks. Some charge as much as three hundred, per review. But the point of automated review is that it shouldn’t cost anything.

Suddenly the cost proposition that swung the business case doesn’t work: you assumed you were saving an hour legal’s time, which you unitise at, say, $250. But that is a nominal cost. It is sunk.

So, unless you can prove that your contract review tool will make some of your lawyers redundant, it is not saving you any real money. It is costing real money — more than it would for your internal team to just knock off the confi in the first place.

It’s not really reg tech at all. It’s just outsourcing that you’re calling reg tech

Reg tech that doesn’t work without a human standing behind it to prop it up isn’t reg tech: it is a gimmick. It is a form of disguised outsourcing. But you are outsourcing to a black box: to personnel you can’t see, you didn’t hire and you can’t evaluate, but who must be cheaper than your own personnel, since you are now paying for them, this confounded reg tech, and the rent your software as a service provider is extracting on the whole operation. If the reg tech checkers are better than your own people, hire the reg tech firm as a recruitment consultant, not to provide crappy AI.

It makes for more work downstream

Now this is not so much a function of the technology but the perverse incentives that operate inside a sprawling organisation. Bear in mind an employee’s primary driver is covering her own arse first, then, where time allows, her organisation’s. There are two places where legal decisions need to be made: Firstly, in configuring the playbook that supplies the parameters for the contract review tool; and secondly, by the legal eagle herself, in combat during a live negotiation, when confronted with a challenge from the counterparty.

• The playbook: There is not a negotiation manual nor a playbook on the planet which stipulates walk-away points at an actual, real-life, point at which the organisation will actually walk away. Not a one. It is wrought in the abstract, without the benefit of individual mitigants that might accompany any project. It is modelled not on the golden mean, but the lowest common denominator. It will have too many rules, and all will be informed by the credit officer’s refrain: it can’t hurt to ask. They may even employ that fatuous supposition that some justify by reference to behavioural economics, that it leaves us something to concede so that the client can think it has won something.
• The negotiation: it is a great comfort and solace to an inhouse lawyer to be able to make commercial decisions, to concede technical or finicky points, and let de minimis points go, on the fly. This is what gives the legal eagle her wings. This vouches safe her autonomy, her mastery and her purpose. This is why she shows up for work. This is why she slogged through all those interminable lectures about promissory estoppel all those years ago. There is something ineffable, even infinite about that knowledge: it is impervious to measurement; it lies in a rich forensic magisterium beyond the censorial gaze of internal audit. You cannot quantify it. In this sunlit realm, we legal eagles can truly fly.

Immediately, the playbook interferes with the free run of our young eagle’s magisterium. Before she can even unfold her wings, the contract review tool has marked-up the document to rule, faithfully cleaving to each enacted stricture of the playbook, however prudish, tiresome or nonsensical. It is like Horton the goddamn Elephant, doggedly sitting on a nest up a tree. Now bearing in mind that saving her precious time is of the essence, it would be most counterproductive for our young eagle now to undo all those fussy, machine-generated corrections — and so we see we have created a cyber version of our old favourite the anal paradox. And, paradox aside, why is de-stupiding a machine’s output any more valuable than de-stupiding the original draft in the first place? Did that cost get baked into the business case?

The measure of a legal eagle

But it is worse than that: our legal eagle is snookered: her ineffable judgment is, courtesy of this fusspot machine, suddenly measurable. The bureaucrats can run MIS reports. They can see what she is doing. They have a yardstick, and it will tell them things like, “Lauren Eagle departed from the recommended standard agreement term seven times in June 2019” The rule of self-preservation, which overrides an abstract yen of autonomy whatever Daniel Pink might say, discourage deviation from guidelines, even though the guidelines are stupid. But, dilemma! If she doesn’t intervene, her client is liable to explode into an incandescent rage at her insistence on a two-year term when he expected three. In any case, she can expect to be knee-deep in negotiation, hastily-convened conference calls and, after a time, some hard-tack reverse-ferret client relationship management on a point which, had she not bothered with the contract review tool, she would never have raised at all. Look — it is only a confi, and it will get sorted out, but remember the original point of the contract review tool was to cut down on work and save costs, not generate more if it.

There is a role for CRM

If you step back and re-contextualise the promise of reg-tech, there is a role for automated contract review. But it is not to supplant humans, but just to give them a scratchpad. If the AI does work at all and you can take out the human backstop, and throw anything at the software, and commit to training it as you go, then you might find it starts to be useful after a while. A machine that can pick out indemnities, governing law, limitations of liability and exclusions in a 40-page contract— just highlighting them, even if not having anything much to say about them — that would be a useful contract companion. This is a much less ambitious, but more realistic, goal.

See also

• Why is reg tech so disappointing?
• A World Without Work