Contract analysis

Luminaries, thought leaders and digital prophets will tell you that machines can now read and annotate contracts, such that yon poor legal eagles are no longer needed and will shortly be looking for work as pleasure droids or something. Quoth a such digital prophet:

The Devil’s Advocate™ →Go uptown Sir! Sir! I've found an indemnity! In which the curmudgeonly old sod puts the world to rights. Index — Click ᐅ to expand: The Devil’s Advocate Tell me more Sign up for our newsletter — or just get in touch: for ½ a weekly 🍺 you get to consult JC. Ask about it here.

Besides manipulating the physical world, machines are also increasingly encroaching on tasks that, until now, have required a human ability to think and reason. In the legal sphere for example J. P. Morgan has developed a system that reviews commercial loan agreements, it does in a few seconds it does what would have required, they estimate, about 360,000 hours of human lawyer time.

— Daniel Susskind, A World Without Work

Sounds fantastic, doesn’t it? Preternaturally intelligent silicon minds scanning and processing gigabytes of diverse text in an instant and analysing it for all material quirks and issues, like Zen from Blake’s Seven. Get your coat, legal eagles.

Data extraction tools

But let’s just stop and consider what is actually going on here. J.P. Morgan reviewing, presumably tens and probably hundreds of thousands of commercial loans (I am allowing JPM’s legal eagles, for whom I have great respect, ten hours to review each loan.)

These loans will be rendered in the hideous, overwrought prose of the banking lawyer, to be sure, but — even allowing for template variations and evolution (by which I mean periodic Cambrian explosions of flannelry — to which not even the House of Morgan is immune), the contracts will all be basically the same. There will be a predictable, schedule of customer details and economic variables — dates, amounts, currencies, rates, and optional elections — but the legal terms will be homogeneous.

So, “reviewing one hundred thousand commercial loan agreements” really means processing one hundred thousand tables of variables. This, doubtless, would take an aeon if assigned to a battalion of legal eagles, and would in the process drive each of the poor saps up the wall, but to be clear it would require absolutely no judgement, let alone legal acumen. It is a matter of transcription.

Assigning a lawyer to this task would, in any era, have been an act of cruel & unusual punishment, not to mention economic folly. It is not news that difference engines can process information better than meatware. JP Morgan’s sensible use of information processing power is hardly a horseman on the lawyer’s ridge.

Contract review tools

More potentially transformative are emerging “artificially intelligent” contract review tools. These promise to take a contract draft received from a counterparty, and analyse it against a pre-defined playbook, together with other examples which the machine has already learned from, mark it up, and return it for completion of the negotiation. A common subject for these contract review tools is confidentiality agreements.

But even here the technology disappoints and, in predictable ways, makes an existing problem even worse than it otherwise might be. Institutions are decommissioning review systems the same way they are repatriating once offshored operations functions. It is worth investigating why.

The problem statement

Start with the problem the contract review tool is designed to solve: Confidentiality agreements are seen as generally low-importance, low-risk agreements — assuming they are properly negotiated — that one must get through as quickly as possible in order to get on with more fruitful commercial negotiations. They tend to be “I’ll show you mind if you show me yours” kind of an affair. But — especially in this data obsessed world — they are buried risks if you don’t watch them carefully. The JC has a whole confi anatomy you can peruse if you would like to know more.

So you need your legal eagles to be en guard to stop the stupid getting in.

Now: confidentiality agreements come in all different shapes and sizes. They needn’t be longer than a few paragraphs, but our American friends are given to presenting 15-page bunker busters, which in the main amount to much the same thing as a concise one, but you never know and you must watch them like a hawk. There are a few points you need to check: the definition of confidential information, exclusions from the general definition, rights to disclose information that is confidential, the term, scope of remedies for breach. It is all, in concept, standard stuff. But it is a faff — depending on how excruciating is the writing style of the author, it might take up to an hour to review, correct mark up and send back. And it isn’t exactly glamorous.

So the problem: it’s slow, it takes a bit of time, the review parameters are fairly complicated (but not complex) — there is a limited risk of missing something — and ones inhouse legal eagle has invariably got better things she could be doing. This is, in person hours, effectively costing the firm money.

The contract review tool as a solution.

The contract review tool promises to perform this first basic check by reference to a pre-defined playbook of confidentiality policies, rather like a triage unit at a military hospital. You give it a once over and it goes out the door. Brilliant.

But it isn’t as simple as that

But it turns out parsing text lovingly confected in the brow of some unknown legal eagle isn’t as easy as all that. Just boring syntactical things like handling plurals and irregular verbs is extremely hard to code. And while in time the machine will get better at that, the universe of possible ways of articulating an idea is infinite and, while most legal eagles suffer from a form of locked-in syndrome which inhibits access to large parts of their imaginations when it comes to exercising their talent for textual complexity and saying things which, for the avoidance of doubt, are not in doubt, it is an information superhighway. Parsing the fevered prose of a human lawyer will always need some kind of a human sense check. So, the contract review tools build in exactly that kind of sense-check function. They hire paralegals, in low-cost jurisdictions, to check the output before sending it back. This has three unfortunate consequences:

It is slow

Firstly, it slows down the output: Instead of getting your markup instantaneously, you get it three quarters of an hour or more later. This is more than enough time for the modern eagle to have been comprehensively distracted by something else, and in any case it is longer than it would have taken to review and fix the contract in the first place.

It is more expensive

Secondly, it adds to the cost. Now, to be sure, reg tech providers are master rent-seekers anyway, but here they have an actual out-of-pocket cost which they have to pay. Thus, the contract review tool will carry a heavy charge per document review. Better ones charge less than a hundred bucks. Some charge as much as three hundred, per review. Suddenly the cost proposition that swung the business case looks a bit squiffy: your starting assumption is you are saving an hour of your internal legal’s time, which you unitise at, say, $250. But that is a nominal cost. It is sunk. A new confi coming in doesn’t generate that cost, and firing the thing out to your contract review tool doesn’t save it. Internal legal eagles are a fixed cost, and are notoriously hard to shred back to the business. They just sit there, on the firm’s dime, whether you use them or not. Most work hard, of course — the legal eagle who punches out at 5pm on the smacker on any day, let alone every day is — well, a rare bird. He will look at that confi, and anything else that needs to get done, at some stage during the day. Legal eagles don’t work to rule. So, unless you can prove that, with your contract review tool you can actually let some of your lawyers go, it is not saving any money. It is costing money — more than it would have cost your internal team just knock off the confi in the first place.

It isn’t actually reg tech at all. It’s just outsourcing that you’re calling reg tech

So reg tech that doesn’t work satisfactorily without a human standing behind it to prop it up isn’t reg tech: it is a gimmick. What we are really talking about here is outsourcing. But to a black box: these are personnel who you can’t see, you didn’t hire and you can’t evaluate, but you know for sure are a lot cheaper than your own personnel since you are now paying for them, this confounded reg tech (also a black box, by the way: how much of the work-product is the tech, how much the meatware? If they won’t show you, assume it is mostly the meatware), not to mention the rent your software as a service provider is seeking to extract on the whole operation.

It makes for more work downstream

Now this is not so much a function of the technology but the perverse incentives that operate inside a sprawling organisation. Bear in mind the primary driver of most employees in risk and control functions is covering their own arses first, then, where practicable, their organisation’s. There are two places where legal decisions need to be made: Firstly, in configuring the playbook that supplies the parameters for the contract review tool; and secondly, by the legal eagle herself, in combat during a live negotiation, when confronted with a challenge from the counterparty.

• The playbook: There is not a negotiation manual nor a playbook on the planet which stipulates walk-away points at an actual, real-life, point at which the organisation will actually walk away. Not a one. It is wrought in the abstract, without the benefit of individual mitigants that might accompany any project. It is modelled not on the golden mean, but the lowest common denominator. It will have too many rules, and all will be informed by the credit officer’s refrain: it can’t hurt to ask. They may even employ that fatuous supposition that some justify by reference to behavioural economics, that it leaves us something to concede so that the client can think it has won something.
• The negotiation: it is a great comfort and solace to an inhouse lawyer to be able to make commercial decisions, to concede technical or finicky points, and let de minimis points go, on the fly. This is what gives the legal eagle her wings. This vouches safe her autonomy, her mastery and her purpose. This is why she shows up for work. This is why she slogged through all those interminable lectures about promissory estoppel all those years ago. There is something ineffable, even infinite about that knowledge: it is impervious to measurement; it lies in a rich forensic magisterium beyond the censorial gaze of internal audit. You cannot quantify it. In this sunlit realm, we legal eagles can truly fly.

Immediately, the playbook interferes with the free run of our young eagle’s magisterium. Before she can even unfold her wings, the contract review tool has marked-up the document to rule, faithfully cleaving to each enacted stricture of the playbook, however prudish, tiresome or nonsensical. It is like Horton the goddamn Elephant, doggedly sitting on a nest up a tree. Now bearing in mind that saving her precious time is of the essence, it would be most counterproductive for our young eagle now to undo all those fussy, machine-generated corrections — and so we see we have created a cyber version of our old favourite the anal paradox. And, paradox aside, why is de-stupiding a machine’s output any more valuable than de-stupiding the original draft in the first place? Did that cost get baked into the business case?

The measure of a legal eagle

But it is worse than that: our legal eagle is snookered: her ineffable judgment is, courtesy of this fusspot machine, now quite measurable. The bureaucrats can run MIS reports. They can see what she is doing. They have a yardstick, and it will tell them things like, “Lauren Eagle departed from the recommended standard agreement term seven times in June 2019” The rule of self preservation, which overrides an abstract yen of autonomy whatever Daniel Pink might say, discourage deviation from guidelines, even though the guidelines are stupid. But, dilemma! If she doesn’t intervene, her client is liable to explode into an incandescent rage at her insistence on a two year term when he expected three. In any case, she can expect to be knee deep in negotiation, hastily-convened conference calls and, after a time, some hard-tack reverse-ferret client relationship management on a point which, had she not bothered with the contract review tool, she would never have raised at all. Look — it is only a confi, and it will get sorted out, but remember the the original point of the contract review tool was to cut down on work and save costs, not generate more if it.

There is a role for CRM

If you step back and re-contextualise the promise of reg-tech, there is a role for automated contract review. But it is not to supplant humans, but just to give them a scratchpad. If the AI does work at all and you can take out the human backstop, and throw anything at the software, and commit to training it as you go, then you might find it starts to be useful after a while. A machine that can pick out indemnities, governing law, limitations of liability and exclusions in a 40 page contract— just highlighting them, even if not having anything much to say about them — that would be a useful contract companion. This is a much less ambitious, but more realistic, goal.

See also

• Why is reg tech so disappointing?
• A World Without Work