Template:M intro stats future-state uncertainty

In his 2024 Christmas essay for Nature magazine Professor David Spiegelhalter asks the reader to imagine a coin-flip and guess the probability of it coming up “heads”.

No surprise: it is “50:50”.

Professor Spiegelhalter then flips his imaginary coin, catches it, covers up the outcome and asks: what’s the probability it’s heads now?

The answer is, of course, still 50:50. But the nature of the contingency has changed. Before the flip, we were dealing with an undetermined future: we did not know the outcome because it had not yet happened. After the flip, the event has a value, but we don’t know what it is.

Statisticians call the first such state an “aleatory” contingency, but let’s bust that jargon and call it a “future-state contingency”. This is something we can’t be sure about because there is nothing yet to be sure about; only the mathematical, yet artificial, narrative of probability.

Artificial? Coin flips, dice rolls and other games of chance only work as they do because, by design, they feature a “well-defined probability space”. Their “prediction environment” is artificially bounded, predictable, independent and stable, so we can calculate an expected value for each possible future state.

We design these environments to exclude weird, non-linear or unexpectable events: when you flip a coin, the outcome cannot be a bowl of petunias.

A well-defined probability space is thus a tame, not wicked environment. It is a “nomological machine”:

“A fixed (enough) arrangement of components, or factors, with stable (enough) capacities that in the right sort of stable (enough) environment will, with repeated operation, give rise to the kind of regular behavior that we represent in our scientific laws.”

So, in a well-defined probability space, the expected value for an event is the sum of each possible outcome multiplied by its probability. For a fair coin (where heads is 1 and tails is 0) it is 0.5. Even though the events have not yet happened we confidently expect their average valued to converge on the expected value over a large number of flips.

Present-state contingencies

Staying in our tame, well-defined probability space of coin flips, let us turn to the unknown present. Now, after a flip, but before we discover its outcome, the contingency is personal: the universe has a determined state, it is stable, we just don’t know what it is. Statisticians call this “epistemic” contingency, but we will call it “present-state contingency”. This is still a situation of risk and not uncertainty.

Terminology check: contingency, uncertainty and risk In this article I will be talking about “risks” and “uncertainties” in the sense introduced by Frank Knight in 1921. A situation of risk arises in a “tame” environment where all potential outcomes are known and their probabilities are calculable: a “well-defined probability space” as described below. A situation of uncertainty arises in a “wicked” environment where potential outcomes are unique, interdependent, unknown, non-linear or unprecedented, and their probabilities, therefore, cannot be calculated. Since I argue that “risks” and “uncertainties” sometimes resemble each other, I have used the ungainly word “contingency” to mean a situation that may be either. Who does not love a definitions section?

For a well-defined probability space a present-state contingency carries the same expected value as a future-state contingency. The chance of that coin being heads remains 0.5, before and after it is flipped until its outcome is known. It does not matter whether the flip happened yesterday, happens later today or not for another five years: the odds it will be “heads” are still 50:50. No surprise: this kind of stability is one of the criteria for a well-defined probability space, after all.

Nonetheless, conceptually, uncertainties about the present and the future are very different. One is about the state of the universe itself, the other is about what we know about the state of the universe. They are only the same in a well-defined probability space because we have, well, defined it that way. Most uncertainties in life arise in “probability spaces” that are not well-defined. Here, present and future probabilities are different.

Philosophical side-bar

A quick sidebar for those, like me, still scarred by “free will and determinism” lectures they took in Stage I Philosophy.

If we know the universe to be but an intricate clockwork where the future can be deduced and its path fully determined, then all probability spaces are well-defined, and “future-state” and “present-state” uncertainties are the same. That we have not managed yet to calculate the expected value of everything only reflects a lack of data, processing horsepower or gumption.

But if the universe is not pre-determined — if there is free will, or if it is not conceptually possible to calculate the future — then it is not a well-defined probability space and “future-state” and “present-state” probabilities are different.

Many apparently learned and sensible people believe the universe is determined. JC does not. However since it is conceptually impossible to know for sure, we should treat it as undetermined.

So, let us waste no more time speculating about the futility of worrying about something it is futile to speculate whether we should be worrying about.

The difference between present-state and future-state uncertainties

So, in an undetermined universe, beyond the narrow class of well-defined probability spaces, what is the difference between future-state and present-state contingencies? These are not like coin-flipping competitions: we are dealing with non-linear, complex systems. Bowls of petunias may arise: we know this because, throughout geological history, they have done.

Here the range of possible outcomes is indeterminate, events may be interdependent and boundaries and rules are unknown. It is an infinite game, and somewhere between hard and impossible to predict what will happen in it, depending on how much we know, and how far out we are projecting.

Take the question:

“Who will be UK Prime Minister in the future?”

It depends on the time horizon. Over 100 years, it is impossible to predict. None of the realistic candidates have yet been born. The role may no longer exist: for all we know the UK of a century from now may be governed by A.I. It may not exist at all: its current version dates only from 1922, so it is not unreasonable to expect it to reconstitute again in a similar period. So, stupid question.

It seems less stupid to predict over a short time horizon. This seems a fair question:

“Who will be UK Prime Minister tomorrow?”

We can be quite confident that the role will exist, the UK will exist and the person occupying it will be the person who occupies it today. Even at a point of maximum uncertainty (on the eve of a General Election or a no-confidence vote), there are only two or three realistic candidates. We can assess their chances with the information we have. While there remain outlying “unknowns” — the planet could be wiped out by a solar flare, or polling could be wildly wrong — beyond these, the range of possibilities is very limited and quite assessable.

So assigning probabilities to future states seems reasonable on two conditions: firstly, confidence about the true present state of the system — that our epistemic certainty is high — secondly, that our forecast period is so short that the system has no time to “misbehave”. As long as we can expect a complex system to act like a well-defined probability space, we can treat it as if it were one.

The longer the time horizon, the more our confidence wanes, but ineffably: we cannot calculate robust expected values for future states of a complex system. The best we can say is “Our confidence level starts near 100% for the “present state” and as the time horizon for the “future state” elongates it falls to 0% (after say 100 years), but we can’t calculate the expected value, or plot the shape of its decline, at any time in between”.

Present states of complex systems

Our uncertainty about the present state of a complex system is qualitatively different from our uncertainty about any of its future states.

Just as for Professor Spiegelhalter’s covered-up coin flip, the present-state contingency is “epistemic” and not “aleatory”: the system’s state is settled; only our knowledge about it is lacking. But this time it is a situation of uncertainty, not risk.

In a well-defined probability space, we know all relevant parameters. In an ill-defined probability space, we don’t. There are known unknowns and unknown unknowns. Our knowledge of the present state is markedly less complete than it is for the present state of a coin-flip. We do not have enough information to calculate statistical probabilities. The best we can do is to build narratives and work out narrative probabilities from there. These depend on our chosen narrative being correct. Sometimes we can be fairly confident:

“Who was the British Prime Minister on 31 August 1888?”

The criteria for this question are straightforward, and the records we have of them are public and complete and our narrative is robust. The official appointment letter from the Monarch formally appointing each Prime Minister is preserved in the National Archives. It is hard to see how there could be much doubt about it.

But what about:

“Who was Jack the Ripper?”

There is no letter in the National Archives. We can build a narrative and test it with the evidence we have, but all hypotheses are weak. It was likely a male living in or near Whitechapel who could not account for his whereabouts on 5 nights over six weeks in the autumn of 1888. But this does not narrow it down by much. A few men have been singled out but the likelihood, on the information we have, that it was any one of those named suspects over “the class of all males living around Whitechapel in September 1888”, is low. Hence, perfect conditions to vouchsafe the future of the Jack the Ripper speculation industry.

Probability is all there is

We are in “Who was Jack the Ripper?” territory more often than we think. Often, these non-statistical, “narrative probabilities”, weak as they are, are all we have. The perpetually fractured state of public debate suggests we are all more confident in our own assigned probabilities than we should be.

We are pattern-matching machines. Instinctively and automatically, we pattern-match to our internal narratives — what we used to call our “cultural baggage”. We manage and fit new information as it emerges against that model.

In the early stages, we might adjust or reject the model to reflect contrary information. But the more validating information comes in, the more committed we become to our model and the less inclined we are to accept information that seems to falsify it. We will distinguish the salience of conflicting new information to protect our model.

We all do this: our worlds would quickly fall apart if we did not. We can be quite imaginative about it. It is a kind of well-intended confirmation bias, though it can lead to prosecutor’s tunnel vision.

But still: it is an unavoidably subjective approach. We can only judge based on mental models generated from our personal experience. We are always prone to rationalising information we were not expecting: to a man with a hammer, everything looks like a nail.

Perhaps we might look at everything not so much as a nail but as a narrative probability. This involves letting go of a certainty many of us find strangely comforting, but the dividend is flexibility and adaptability, which, in an infinite game, is a tool worth having.

Traitors and probability

Here’s a fun way to spoil great human drama: treat defection/elimination games like Traitors, Squid Games or even the board game Secret Hitler, not as games of tactics, subterfuge, detection and betrayal, but simple exercises in basic random probability.

On the one hand, this makes no sense: events in these games are not random. They don’t take place in “well-defined probability spaces”: events and actions, by design, are ambiguous, have multivariate causes and take place in half-lit, human, social spaces. Players have only incomplete information about the rules, motivations, and other players’ statuses.

There is surely an advantage to playing the game well: isn’t there?

Maybe not: experience suggests we are strikingly bad at finding Traitors. Okay but surely applying random probabilities would be worse?

Our misplaced belief that we have the gumption to beat a random guess is a perennial human flaw. It propels the asset management industry, after all. Sports franchises have long since embraced Moneyball.

So, allow for a moment that, in a “Traitors” scenario, we can’t beat the odds. If we played the game as if it we had no more than a random chance, what would it look like?

Less alluring to play: a 1/24 chance of winning at random doesn’t sound like much chop. And less entertaining to watch: fewer tears, fewer misplaced reflexive pronouns and no traduced allegiances or heartfelt declarations that “I’m faithful, 100 per cent.”

If you really wanted to win, this is what you would do. They may not be perfectly formed probability spaces but they are bounded and they do operate according to fixed rules (even if their application is obscured).

This operates to re-randomise the non-random events, at least from the players’ own perspective. And here is where “skill” might be a disadvantage. Apparent skill attracts attention; attention attracts (bad) heuristics and bad heuristics lead to ejection. The murder/banishment process systematically selects skilful, or at least active players for early elimination. (Maybe the skill of an elimination game like this is to seem not to have skill?

Now: if X murders Y, but Z knows neither who murdered Y nor why, then as far as Z is concerned, Y’s murder is random. In the game, what matters is not what actually happened, but what Z believes happened.

This is, needless to say, deeply subjective: the great insight of Bayesian reasoning is that to a person operating in a social space — that is, all of us, all of the time — what matters is personal uncertainty and not just factual indeterminacy. We must always make decisions in ignorance of important facts. We can only do this by informally assigning probabilities.

The decisions we make like this — purblind, biased, misconceived — all the same inform our actions, by which we then change the system. In Traitors, since players are part of the system their behaviour affects the system, and their knowledge prompts their behaviour, this subjectivity is an intrinsic state of the system. Misconception is baked into the environment. We cannot solve it with crystalline logic.

Now, treating Traitors as a random game, prior information tells the players at the outset they each have a 1/24 chance — that is less, than five per cent — of winning. All should expect, with some confidence, to lose.

Their main priority should not be increasing those odds of losing with their behaviour.

Manipulative behaviour courts banishment. Perceptive behaviour invites murder. On the other hand, saying nothing at all may seem manipulative or perceptive in its own right.

Some behaviour — perhaps bumbling affability or skill at missions — might seem benign to both traitors and faithful but as players cotton on to what this behaviour is, they will adopt it, and it should cancel itself out. If everyone displays quintessentially faithful behaviour, no one does. It becomes a random marker. Overly suspicious behaviour also cancels itself out, as the hinky are banished over time.

Any kind of strategic behaviour is liable to get you killed. The optimal strategy is to get as close as possible to non-strategic behaviour as possible.

A way of doing this: declaring yourself specifically to be acting at random, when selecting banishment candidates (and even if murdering as a traitor). This takes all the fun out of it, so much so that you might be banished or murdered as a crashing bore, but it strikes me as a way of optimising your chances of success and best-maintaining relationships!

Are the odds fair, in Traitors, then?

I have battled with finer statistical minds than mine — oh alright, I confess, it was DeepSeek and Claude, and neither of them got it right — but it is pretty plain, intuitively, that traitors have the better of this game, just played on the odds: They are immune from the “murder” rounds, meaning they have less opportunity for eviction, and the rules of the game allow them, and in some cases effectively force them, the recruit new traitors should they run low.