Path-dependent

The JC’S favourite Big Ideas™ →Go uptown Index: Click ᐅ to expand: Big ideas The Devil’s Advocate Philosophy Tell me more Sign up for our newsletter — or just get in touch: for ½ a weekly 🍺 you get to consult JC. Ask about it here.

Path-dependent
/pɑːθ-dɪˈpɛndənt/ (adj.)

Of a circumstance, that its coming about can only be explained by the sequence of events and contributing factors that lead to it; that it cannot be justified or explained by reference only to existing conditions.

The way things turned out depended on the coincidental interaction and juxtaposition of unrelated factors in the ecosystem.

The classic case is of course evolution by natural selection: re run the tape from the beginning and you would not get the same result.

Backtesting, how how “hindsight is a wonderful thing”

Backtesting is a naked way of trying to solve for the future by extrapolating from the past.

It is so naked that these days only the mendacious or the dense have any truck with it. Of course, the mendacious and dense tend to get on pretty well.

The idea is this: collect and analyse years of market data — back through the last meltdown, ideally — and from it construct the optimal trading strategy, that, had one used it, would have returned the greatest profit, and have totally avoided the fallout of that meltdown.

Of course no-one knew at the time how the meltdown would play out, so no-one used this brilliant strategy. For had we known trading patterns would have been different, the data would have come out differently, thereby confounding the strategy.

Backtesting is a preposterous idea.

By the lights of history, the market’s course is now fixed. The data is dead. Inert. But when you are there, in the moment, it is not. It reacts to you, and you react to it. It is dynamic. Symbiotic. Contingent. Alive.

A trading strategy derived from the fossil record of data that was once alive, but is now not, is similarly inert. It is cannot ride, or get swamped by, the symbiosis between market and participant.

This is true of any algorithm that depends on data. Data is history. A photograph. It is good for answering questions to which there already is an answer. (Don’t scoff at this: many good questions are things for which there is already an answer; just one we don’t personally know. Hence: the value of Google. The value of a library.)

But note the mode of discovery: static; historical; final; determinate. Data can tell you how things were.

Causal determinism

We accept “causal regularity” — that science yields truth: that one thing regularly leads to another — because the alternative seems to deny the apparent operation of the universe.

But even here the fossil record flatters to deceive: the lattice of potential causes is far more complex than our wildest dreams — we form those just from what we see and hear — but that is an infinitesimal sliver of all possible events out there. Our histories are works of imaginative fiction. This is why historians do not agree. We make our histories; we do not find them.

By looking at a unitary history (that we made up), in hindsight we miss the contingency from which it was fashioned. Once it is laid down, it looks inevitable. It looks pre-ordained. This is a curiously religious idea.

In any case, even if there is but one past — whether or not we can know it — still there remains, from any given present, an infinity of futures.

The temptation, when we look at such a concrete past, is to see each of the points behind us on that timeline as having determined the remaining history to the present. The extrapolation is that they must determine the future, too. The further back in time a point is, the more momentous it has been in determining our path to here. This seems intuitive: the decisions I made yesterday had little bearing on where I am today: I was already here. The die was long since cast.

But this is not true.

Since that moment thirty-years ago, when you bought that plane ticket to America, you have had thousands of opportunities to buy a plane ticket home again. That you are still in America is nothing to do with that ticket you bought, and everything to do with the tickets home you haven’t bought since.

We have, and our ancestors had, the ongoing ability to change things daily. Everyone makes bad decisions. The key is not to be defined by them. Everyone makes good decisions, too. Keep the good decisions, do what you can to correct for the bad ones.

Our permanent aspiration: from here, make more good decisions than you do bad ones. Improve your ratio. You will not always know at the time. You will learn in hindsight. Iterate.

You can’t undo the decisions of the past, whether made by you or about you, or by your ancestors or about them. You can make different decisions now.

Pragmatist’s prayer and the infinite game

Finite and infinite games is, as ever, a great metaphor for framing these battles of the past and present. For what is a “lived experience”, a “grievance” or a “standpoint”, if not an articulation of history?

The future contains unlived experiences. There are no grievances. Our standpoints, the margins and their intersections are unknown.

Being historical, a lived experience is permanent, and set it stone. It cannot be moved. It cannot be removed. It cannot be compensated for. It cannot be denied. It becomes a monument. A shibboleth. A sacred prophecy. But it is our imaginative construction. We choose our significant events. We build our own memorials. We choose to live beneath their shadows. But our present is a function of every point in the past, not just the ones it's suits us to settle on.

This is the empathetic stance. To adopt a historical narrative: to step into its shoes, to take sides, to exalt it and perpetuate its grievance.

But, look: standpoints iterate. As the present moves through spacetime, we lay down the tracks of future, each new decision we make contributes to our lived experience. We update our standpoints. The decisions of the past for all further away in time and significance. It is an inverse square.

The infinite game counsels us to look at where we are, see what we’ve got and make the best of it. It focuses on the decisions of the now and the possibilities of the future. It regards the past as informational and instructive, not constraining. If I once hit my thumb with a hammer, I know to be careful next time I have a hammer. It does not make me forever a victim of hammer abuse.

The past as a formal system

Not also the idea that the past is a single formal causal chain, that we know about, is is a classic example of legibility in the sense articulated by James C. Scott in Seeing Like a State.

Articulation of history is necessarily a simplification, and model, a boiling down of an infinity of information into a single digestible narrative. It necessarily misses things: relegates things; deems things extraneous. But what the dominant narrative things is important and what the community thinks is important are not the same. In the same way that informal systems and interactions, unseen by the executive actors, are critical to the good order and smooth operation of the state, or a business, so are informal, unobserved, and deprioritised interactions fundamental to history. Models lie.

So not only is “the past” — as we articulate it — inert and immutable, it’s not even true. We can, and do, adjust it by changing our account of it. History is written by the winners — but it is a long game and just who is the winner is prone to change. Those who dominate the narrative from time to time decide who the winners are. This is the Orwellian concept: write and re-write and erase our history, whilst insisting on its utter continuity. It may be so that the external history of the universe is immutable — but we have no access to that. We wouldn’t recognise it even if we had it. We can only tell ourselves stories, and they have internal meaning, but no transcendent one.

See also

• Empathy and compassion
• Evolution