Template:M intro letby Letby and statistics

JC does not know whether Lucy Letby was, in fact, responsible for any, or all, of the incidents for which she was convicted. A rare point of consensus is that other than Ms. Letby herself, no one at this remove can possibly know.

What interests me is how the criminal justice system, the criminal bar and the establishment — under pressure as they rarely have been in recent times — react to the events as they unfold. Things feel like they are gathering momentum. Yet still there are holdouts.

There is plenty of coverage of the new evidence against Ms. Letby’s conviction: it is more interesting to review the strengths of the arguments that continue to be advanced that the conviction was safe.

The first, to sweep away the infamous “Texas sharpshooter” critique, is that “Lucy Letby’s prosecution had nothing to do with statistics”.

The Texas Sharpshooter

Schafer said that he became concerned about the case when he saw the diagram of suspicious events with the line of X’s under Letby’s name. … The diagram appeared to be a product of the “Texas sharpshooter fallacy,” a common mistake in statistical reasoning which occurs when researchers have access to a large amount of data but focus on a smaller subset that fits a hypothesis.

Quick refresher: The “Texas sharpshooter” is a famous fallacy of probabilistic reasoning, first connected to Ms. Letby’s case in Rachel Aviv’s, New Yorker investigation of 13 May 2024. It is a common form of confirmation bias we fall into when interpreting statistics:

A drunk Texan farmer fires 100 rounds blindfolded at the side of this barn. Later, an inspector passes the barn and notices a spot where, by chance, five bullets hit close together. She paint a targets around the cluster and declares: “This is the work of a master sharpshooter”.

The fallacy is that she is working backwards. Ignoring other bullet holes (and those she couldn’t see, that missed the target altogether!) she found a pattern after the fact and assumed it was an intended target. She made the data fit her conclusion, rather than drawing conclusions from properly collected data.

The notorious spreadsheet rota, picking out 25 shifts out of a thousand or more shifts, being the ones for which Ms Letby was charged seems to shriek Texas Sharpshooter, so prosecution supporters repeatedly stress that this case was not about statistics. The spreadsheet was a de minimis part of the story, admitted by consent. The Texas Sharpshooter is irrelevant.

There are two flavours to this argument: firstly, neither side ran an argument based on statistics or probabilities. Statistics were, therefore, never in contemplation.

Secondly, the key evidence implicating Ms. Letby was not statistical in nature. There was direct evidence that she committed the crimes: therefore any statistical speculation, and all this Texas sharpshooter nonsense, is beside the point.

No one mentioned statistics

I can’t get the statisticians out of their ivory towers: this is where they just don’t understand what the prosecution case was about. There was a consistent pattern: first of all, the death was unexpected; secondly, the death was unexplained; and any neonatologist will tell you that it is very very rare indeed for a baby on a neonatal unit to suddenly collapse and suddenly die, and where the death is unexplained.

The Crown’s “star witness” Dr. Dewi Evans’ exasperation with “statisticians in their ivory towers” is plain to see. Nonetheless, he then appeals directly to statistics himself: “It is very, very rare indeed for a baby on a neonatal unit to suddenly collapse and suddenly die, and where the death is unexplained.”

Whether he is drawing on his vast professional experience, or referring to officially published metrics, Dr. Evans is inferring his conclusion from an asserted historical rarity. That is an appeal to probabilities. It is to say, this explanation is so unlikely that there must be an alternative, more probable explanation for the collapses.

That is a probabilistic inference: Given that it is rare for babies to suddenly collapse and die without explanation, and given that several babies did suddenly collapse and die without explanation, and given that one nurse was present at all the collapses, the most likely explanation for the collapses is that they were caused by that nurse.

Dr Evans infers that a malicious actor is the most likely explanation.

The crown did not rely on statistics

Alternatively, it is said even though probabilities are theoretically relevant, that was not the crown’s case. The clinching evidence was other, non-inferential evidence — in the vernacular, “posterior information” — that unequivocally implicates Ms. Letby, and overrides the baseline improbabilities. However unlikely a perpetrator Ms Letby might have been in the abstract, there is evidence that it was her in fact.

The first thing to say here is that of the expert medical evidence put before the court — Dr Evans’s entire contribution, that is — squarely none of it specifically implicates Ms Letby. There is no evidence that, if there was an air embolus, it was Ms. Letby who administered it. Nor the insulin, nor the feeding tubes nor the alleged physical trauma. No evidence attributed any of these events personally to Ms. Letby. The only connection to Ms. Letby was by dint of her alleged presence on the ward.

In O.J. Simpson’s murder trial, a key defence argument was that the statistical evidence about DNA found at the crime scene — which suggested a one in four million chance it was not Mr. Simpson’s — implied the DNA could belong to several other residents of a city the size of greater Los Angeles.

So the argument went: if Mr. Simpson was just one of a small number of Angelenos who could have left that DNA, there must be some reasonable doubt it was Simpson who actually did leave the DNA.

This ignores key posterior facts: only one of those theoretical candidates — Mr. Simpson — was married to, and had a history of violence against, the victim.

Those posterior facts made the possibility that the DNA sample could, theoretically, also match a handful of other unidentified people, in a densely populated thirty-thousand square mile area meaningless.

Does the same hold for Ms. Letby? Can direct facts displace evidential improbabilities?

They might, if there were any. If eye-witnesses saw her dislodging feeding tubes, injecting air, or adulterating feeding bags with insulin, but they did not. There is no direct evidence the victims were murdered at all.

All the evidence, as to murder, and as to Ms. Letby’s alleged role in it, is weakly “circumstantial”. Circumstantial evidence can only make the proposition to be proved more or less likely: it does not directly attest to it.

Of this circumstantial evidence, the only part that specifically implicates her, as opposed to “malice by someone unknown”, is her alleged opportunity: she was, it is claimed, present whenever untoward events happened.

But if that is the argument, we have not dispensed with the role of statistics.

And this is all beside the point because the Crown’s case was explicitly predicated on statistics. They may not have been well-argued, but this is exactly how prosecutor Nick Johnson KC opened the case against Lucy Letby on 10 October 2022:

“Prior to January 2015, the statistics for the mortality of babies in the neo-natal unit at the Countess of Chester were comparable to other like units. However, over the next 18 months or so, there was a significant rise in the number of babies who were dying and in the number of serious catastrophic collapses. And these rises were noticed by the consultants working at the Countess of Chester and they searched for a cause.”

[…]

“Having searched for a cause, which they were unable to find, the consultants noticed that the inexplicable collapses and deaths did have one common denominator. The presence of one of the neonatal nurses and that nurse was Lucy Letby.”

This case was absolutely about statistics.