Backtesting

The Jolly Contrarian’s Glossary The snippy guide to financial services lingo.™ →Go uptown Index — Click the ᐅ to expand: Glossary Tell me more Sign up for our newsletter — or just get in touch: for ½ a weekly 🍺 you get to consult JC. Ask about it here.

Here's what investopedia has to say about backtesting:

Backtesting is the process of testing a trading strategy on relevant historical data to ensure its viability before the trader risks any actual capital. A trader can simulate the trading of a strategy over an appropriate period of time and analyze the results for the levels of profitability and risk.

Which makes it sound sensible, measured, prudent and not the systematically moronic idea it is.

Backtesting is one of those cognitive dissonances that has passed, for now, into embarrassed history, but is sure to re-emerge the moment the markets regain their giddily optimistic resting state. Markets are fueled by the self-confident delusions of vain men.

Once upon a time a resourceful salesperson with a clever new product (one that, you know, embedded leveraged alpha, or something) had to go through the motions of justifying its “extraordinary potential” when pitching it to his marks.

The problem with financial markets was, is, and always will be this: you can’t anticipate the future. This isn’t a shortcoming of contemporary risk management theory: it’s in a market’s very nature. If you could, you couldn’t make money betting on it.

You know the story about the frog and the scorpion, right?

So these randy salespeople needed something their clients could take back to their risk committee to demonstrate the rigorous financial analysis™ that had gone into their product design™. Then a lightbulb moment: backtesting. There may be no data from the future, but there’s buckets of the stuff from the past. Here your Bloomberg terminal is your friend. All you need are ninja Excel skills — every blighter has those — and you can compare how your ingenious strategy would have done against benchmarks and market averages had you run it for the last five years.

Which you haven't been doing, because you only just thought it up, because this strategy never would have occurred to you had you not known how the markets had actually behaved over the last five years. There was no data from the future in the past.

And how would it perform? Unless you are a bigger knucklehead than your client, SPECTACULARLY. Whatever the weather. You designed it, with 20:20 hindsight, specifically to look good against this exact data set, remember? However mendacious you might be, the ruse is so obvious that no client who entertained it for a moment deserves any sympathy at all. Yet in their thousands, they did.

They worked it out eventually. The hard way.

Thanks to the “chart” function in Microsoft Excel could render these illustrations in animated multicoloured, three-dimensional graphs, splatter analyses. In every case the strategy outperformed beta and any other indicator in the market that the salesperson cared to represent. But the restaurant booking was in twenty minutes, so the client had seen all he needed to see.

See also

• Confirmation bias
• Snake oil