The Unaccountability Machine

The Jolly Contrarian’s book review service™ →Go uptown The Unaccountability Machine: Why Big Systems Make Terrible Decisions - and How The World Lost Its Mind — Dan Davies, 2024. Index: Click ᐅ to expand: Book reviews Tell me more Sign up for our newsletter — or just get in touch: for ½ a weekly 🍺 you get to consult JC. Ask about it here.

’Tis neither malice, spite, nor virtue
Whose ledger swells, or plucks, the seedy fruits of progress —
But mainly accident.
Lest thee with surety know aught else —
Withhold thy assignations.

—Otto Büchstein, Die Schweizer Heulsuse

This review is sent from an unmonitored account. Please do not reply.

—Anon.

On a lack of barking dogs

The epic judicial processes of 2024 have been Tom Hayes’ appeal against LIBOR rigging, about which we have had much to say elsewhere, the Post Office Horizon IT scandal, and the curiously unfolding non-legal process of re-evaluating the safety of Lucy Letby’s conviction. All three are resolving to the question: to what extent can we put this absolute shower down to the nefarious, or just bone-headed, interventions of individual operators?

For much modern of modern business management — you hardly need an advanced degree in operations research to know that, these days, there’s a lot of it — exists specifically to prevent bad apples, or stupid apples, subverting our complex modern systems. Its record catalogues a singular failure to achieve that basic end. Our roll of honour refers. LIBOR rigging and the sub-postmasters débâcle are but pinnacle examples.

With all that infrastructure, superstructure and supervision how were a band of relatively lowly trading staff able to run riot?

With all its infrastructure, internal and external legal advice, consultancy, and, er, second sight, how did no-one stop to think something must be wildly, catastrophically, wrong with the Post Office’s basic theory of the situation? How did no-one, even once, apply Otto’s razor?

Where were all the barking dogs?

Rogue apples, middle England and the grace of God

Either these are peculiar, localised problems — rogue gangs of bad apples plaguing the innocent houses of commerce — or the prevailing paradigm is in crisis and we need a new theory of the game. “Bad apples” are always the preferred diagnosis. They relieve earnest executives of responsibility, leaving at most a deniable residue of blame for hiring the bad apples in the first place. Sidney Dekker is compelling on this.

LIBOR submitters fit the “rogues gallery” identikit nicely: with microscopic adjustments inside an arcane process to which few paid attention and fewer understood, they (allegedly) enriched themselves to the tune of millions of pounds while no-one else noticed. It was almost a victimless crime.

Post office middle managers do not. Few stood to gain from vilifying innocent sub-postmasters, and the emoluments on offer for those who did paled in comparison to the LIBOR submitters’ city bonuses. These people did not seem psychopathic — unreflective, sure; narcissistic, in a few cases; illustrations of the Dunning-Kruger effect, all — but their motivations, beyond observing the Buttocractic oath, were not base. They do not resemble “bad apples”. Their offence — and it is not a crime, though other commentators suggest it may have risen to that level — was weakness and credulity. Lack of spine.

These people are unremarkable, familiar, mediocre middle managers. Over-promoted; over-sure of their own competence; too mindful of nearby censorious trees to appreciate the calumny going on in the wood. Watching their excruciating evidence, three things occur:

First — The weave of life’s tapestry wouldn’t have needed to be that different for these witnesses themselves to have been sub-postmasters on the other end of this outrage. None more so than CEO Paula Vennells, a middle-English lay Methodist, who even looks like a sub-postmaster.

Second — A montage of every utterance, by every witness, of their manifold variations of “I don’t remember” would go for hours. Hail to the all-conquering power of constructive ignorance.

Third — There but for the grace of God go I. Specifically, JC. Post Office in-house legal head Rodric Williams is a fifty-something expat New Zealander. His career trajectory to date, in vector if not altitude, is strikingly similar to JC’s. In the halogen glare of cross-examined hindsight, his ineffectual interventions in an epic miscarriage of justice over an extended period were somewhere between regrettable and outrageous, but none of them resonate as odd. Williams was adept at the sort of pencil-pushing, risk-averse buttocracy drilled by bitter experience into every in-house lawyer in the land. This is what in-house counsel do. This is how we behave.

We should ask ourselves: knowing what he knew then, would we have done anything differently? However tempting it is to take the moral high-ground from the safety of hindsight, we should not kid ourselves here.

Modern corporation as an unaccountability machine

Which brings us, finally, to Dan Davies’ new book. There is, he reports, a crisis of accountability in commerce: the relationship between “we” the general public and “we” the representatives and managers of the corporations who intermediate public life — many are on both sides of this equation, of course — has broken down. This is because modern corporations are designed to diffuse individual accountability for the actions a corporate legal entity takes, using what Davies calls “accountability sinks”. This is Davies’s example:

Someone — an airline gate attendant, for example — tells you some bad news; perhaps you’ve been bumped from the flight in favour of someone with more frequent flyer points. You start to complain and point out how much you paid for your ticket, but you’re brought up short by the undeniable fact that the gate attendant can’t do anything about it. You ask to speak to someone who can do something about it, but you’re told that’s not company policy.

The unsettling thing about this conversation is that you progressively realise that the human being you are speaking to is only allowed to follow a set of processes and rules that pass on decisions made at a higher level of the corporate hierarchy. It’s often a frustrating experience; you want to get angry, but you can’t really blame the person you’re talking to. Somehow, the airline has constructed a state of affairs where it can speak to you with the anonymous voice of an amorphous corporation, but you have to talk back to it as if it were a person like yourself.

An accountability sink arises when we delegate authority over a human process to a rulebook without giving anyone effective power to override it. Airlines, banks, ticketing agencies and online retailers give the public no access to anyone anymore: to confirm our order, or should it change, we are peremptorily notified by email from an unmonitored account. Because query handling takes time and costs money for no gain, the process is designed to impede the instinct to ask questions. By design, we can’t talk back.

The crushing organisational stasis in financial services — and plainly the Post Office — is a variety of the same thing. Even internally, policy, process and the imperative of scale discourage — at the limit, punish — independent thinking.

This design principle explains exactly why no-one at the Post Office saw, or appreciated the significance of or stopped to consider the implications of, the potentially incendiary advice they were receiving. It was not their job to second guess a process that had been set on rails well before they were involved.

In-house lawyers are the sweepers to the curling stones of a firm’s policies and business initiatives. Once a stone is launched, their job is to facilitate its progress, feverishly working away at the ice in front of it to preserve the momentum it already has. It is no part of their role to impede its progress.

In organisations over a certain size there is a presumption, not lightly rebuttable, that others in the organisation know what they are doing. This is its own accountability sink. When a decision appears to have been made elsewhere, whether by application of policy or exercise of someone’s discretion — questioning that decision is to sell a personal put option whose benefit, if there is one, accrues to the organisation, but whose loss allocates solely to you. If you turn out to be right, someone else carries the can. If you’re not, you do. Either way, you have lost a friend.

True, you may head off an exceedingly remote tail risk if some day, years from now, the entire shabby affair is exposed and all participants held to public account before the nation’s watching eyes, but even then if you will be lucky to end up at even money.

In-house counsel is not a moral compass

There is an argument, unstated in much commentary on the case, that the primary role of in-house counsel — of not just the GC when preparing briefings to the board, but all lawyers in the organisation — is to act as the organisation’s moral compass. They, even more than their compliance colleagues, are ideally positioned to sit above the fray, from where they can interrogate the organisation’s baser commercial instincts, at least insofar as they manifest in legal work product.

That’s a plausible theory of the game, but it hardly reflects current practice. For one thing, in-house legal is not in the operational stack, so doesn’t see the great pitch and yaw of BAU activity that animates the firm’s mortal sinews. You wouldn’t situate your moral compass behind the fire extinguishers in the drawer with old chequebooks, broken torches and dead batteries if you intended to ever use it. At most it would be an exceptional function at times of crisis. But it is not even that.

JC has a tongue in cheek history of in-house legal which charts the growth of the in-house legal function from a grey fellow in a cardigan who wordlessly managed the firm’s powers of attorney from a desk by the photocopiers to the weaponised 1,000 strong battle unit we know today.

No part of that transformation grew out of the yen for a stronger corporate conscience. It was all about business facilitation, cost management (legal eagles were meant to be able to call bullshit on their external advisors) and the commodification of legal services. Precisely, the urge to codify processes, set operating rules and take away any need for anecdotal reflection about what the firm was doing or where it was headed. In-house legal is, in this way, an anti moral compass. Legal is an accountability sink machine.

Nor should legal, compliance or any specialised risk function be the firm’s corporate conscience: those values should be imbued in its every one of its representatives, in everything they do. An operating model that supposes legal and compliance to be a crack morality squad, fighting a multi-front war to quell incipient infamy is as ridiculous as it is delusional.

So operationalised has the role become that legal isn’t even expected to prioritise the law. A few years ago, a GC asked her “senior leadership team” for suggestions on how to improve the quality of contributions to the weekly leadership team meeting. JC’s suggestion — “it would be nice if we discussed legal risks every now and then” — earned him a private reprimand. No-one likes a smart-arse. Management reports treat emerging legal risk as a given: they will track progress of central projects and initiatives against time and budget but will have little to say about emerging risks. Dashboards presented to management are expected to show uniformly green — amber and red signifying “behind schedule” or “over budget” rather than “liable to cause $10bn loss or coverage on the front page of the FT.

We wonder how often the name “Archegos” appeared in board papers at Credit Suisse or Morgan Stanley in the months leading up to its implosion. Or 1MDB at Goldman.

Notwithstanding the predictable monotony at which such events keep blindsiding organisations, we are disinclined to think harder about them.

Witness the state of legal thought-leadership in its natural home: LinkedIn. There is any amount halfwittery about legal service delivery, key performance indicators, legal process excellence, diversity and inclusion, demand management and the transformational potential of AI. But good luck finding a think-piece that mentions complexity, non-linearity, systems theory, network effects or Bayesian probabilities as they impact the administration of justice, let alone anything as nebulous as an in-house lawyer’s obligation to make ethical judgments about it.

A confederacy of stupid apples

It is easy, and tempting, to put the Horizon débâcle down to an unusual confederacy of stupid apples — this suits our personal self-esteem because if that is right, such a thing should not happen to us.

We should not be so sure.

“It would be nice,” says a valued correspondent, “if counsel were free to have a working moral compass inside their heads to help take open and ethically sustainable courses of action.”

It would. But — thanks to the unaccountability machine — it is hardly likely. No–one got ahead by calling management out for incompetence.

Bear in mind, too, that servants, including in-house lawyers, are paid partisans. Advocates for the firm. Litigation, in the common law world, is an adversarial process, not a fact-finding enquiry.

Yes, there are standards of disclosure, fair dealing and honesty required but, upon finding weakness in one’s witness, a litigant’s instinct is not instantly to concede the case but to find a better witness. The rules oblige parties to disclose material evidence, but not flakey witnesses. Procedural rules make it hard to call hostile witnesses because you cannot cross examine them.

We are at danger of imputing to these individuals knowledge we now have — courtesy of the ITV drama, and months of cross-examined evidence — but they plainly did not. Not that the Horizon system was flimsy, biddable or prone to errors; they knew that — but that there was no fraud.

It seems absurd. It would be absurd were it not the plain facts of the matter. These people were told, by people they trusted and had no reason to doubt, that the sub-postmasters were fraudsters. That is their operating theory of the game. The fact that the Horizon system was a dog was an irritating fact getting in the way of a nobler pursuit of justice. If your operating theory is that you are on the right side of history — if that is your Bayesian prior — you will filter all incoming evidence that way. You will regard the postmaster who pelts you with intemperate emails about your own incompetence as unhinged. You will think the 15 page typed screed alleging breathtaking fraud against the former NASDAQ chairman as the ramblings of a lunatic. If we have to treat every wild allegation as fact until disproven, we will never get anywhere.

It is hard to know what to prescribe here. Should we set up an adversarial system inside the organisation to mimic the court of public opinion outside it - a “defence department” overtly charged with taking the side of the alleged miscreant? Maybe we should. But

A special mention of the ultimate flimsiness of legal professional privilege here. Some people who really ought to know better put in writing some extraordinary things. The misjudgment seemed so total until you realise that, normally , this class of communications would never see the light of day, barred from view by the deep magic of litigation privilege.

Individuals are, therefore positively disincentivised couraged from raising their hands.

We should not underestimate the overwhelming power of plausible deniability.

See also

• Post Office Horizon IT scandal
• Mark-up the last deal
• There but for the grace of God go I