Backtesting

Here's what www.investopedia.com has to say about backtesting:

Backtesting is the process of testing a trading strategy on relevant historical data to ensure its viability before the trader risks any actual capital. A trader can simulate the trading of a strategy over an appropriate period of time and analyze the results for the levels of profitability and risk.

Which makes it sound sensible, measured, prudent and not the systematically moronic idea it is.

Backtesting is one of those cognitive dissonances that, for now, has passed into embarrassed history, but is sure to re-emerge the moment the markets regain the giddy optimism that is their resting state. Markets are fueled by delusional self-confidence of vain men.

Back in the day, a resourceful salesperson with a clever new product (one that, you know, embedded leveraged alpha, or something) had to go through the motions of justifying its “extraordinary potential” when pitching it to his marks.

The problem with financial markets was, is, and always will be this: you can’t anticipate the future. This isn’t a shortcoming of contemporary risk management theory, by the way: it’s in a market’s very nature. If you could, you couldn’t make money betting on it.

You know the story about the frog and the scorpion, right?

So these randy salespeople needed something their clients could take back to their risk committee to demonstrate the rigorous financial analysis that had gone into their product design. Then a lightbulb moment: backtesting. There may be no data from the future, but there’s buckets of the stuff from the past. Here your Bloomberg terminal is a compendious friend. All you need are ninja Excel skills — every blighter has those — and you can compare how your ingenious strategy would have done against benchmarks and market averages had you run it for the last five years.

Which you haven't been doing, because you only just thought it up, because this strategy never would have occurred to you had you not known how the markets had actually behaved over the last five years.

And how would it perform? Unless you are a bigger knucklehead than your client, SPECTACULARLY. Whatever the weather. You designed it, with 20:20 hindsight, specifically to look good against this exact data set, remember? However mendacious you might be, the ruse is so obvious that no client who entertained it for a moment deserves any sympathy at all. Yet, buy it in their thousands they did.

They worked it out eventually. The hard way.

Thanks to the “chart” function in Microsoft Excel could render these illustrations in animated multicoloured, three-dimensional graphs, splatter analyses. It was brilliant. In every case the strategy outperformed beta and any other indicator in the market that the salesperson cared to represent. But the restaurant booking was in twenty minutes, so the client had seen all he needed to see.

See also

• Confirmation bias
• Snake-oil