Chapter 10: When Machines Fail

Every system in this book works until it doesn't. This chapter is the failure catalog: the documented disasters, the structural risks, and the statistical traps that make backtests lie. A planner insists on this chapter because the question is never "does it work?" It's "what happens when it stops working, and can I survive it?"

Knight Capital: $440 million in 45 minutes

On August 1, 2012, the market maker Knight Capital deployed new trading software to its servers. The deployment reached seven of eight servers. The eighth kept running an old, retired routine called Power Peg, and a new order flag reactivated it.

For 45 minutes, the system bought high and sold low across more than 140 stocks: roughly $3.5 billion in unintended long positions against $3.15 billion short, nearly 4 million executions, almost 400 million shares. About $10 million a minute, evaporating. There was no kill switch. Ninety-seven automated warning emails went unread. The firm survived only through a $400 million emergency infusion led by Getco, was later acquired, and the SEC fined it $12 million.

The industry's lesson, and the one I want you to take, is precise: the automation worked exactly as coded. The design's assumptions failed. Nobody programmed the system to lose $440 million. They programmed a system whose failure mode nobody had fully imagined, deployed it carelessly, and gave it no off switch. Every AI trading system you will ever encounter has failure modes its builders haven't imagined. The serious ones have kill switches anyway.

The Flash Crash: May 6, 2010

A single sell algorithm liquidating 75,000 E-mini S&P 500 futures contracts, about $4.1 billion, on a volume-based schedule with no price or time limits. High-frequency systems passed the contracts among themselves. Volume spiked. The algorithm read the volume as liquidity and accelerated. The Dow dropped nearly 1,000 points and recovered within minutes.

No AI "decided" to crash the market. Dumb automation interacted with fast automation and produced an outcome nobody designed. Now add modern AI to that picture. Former SEC Chair Gensler repeatedly warned that many independently optimized AI systems, all reading the same signals, could trigger simultaneous machine-speed sell-offs no human can interrupt. Research simulations show trading bots learning manipulative, collusive strategies without any human intending it. The flash crash was an accident. The next one might be an emergent property.

The quiet killer: overfitting

The disasters above are dramatic. The one that actually eats most AI trading strategies is boring, invisible, and everywhere: overfitting, building a model that memorized the past instead of learning the future.

Plain-English translations of the four horsemen:

• Overfitting is memorizing last year's exam answers instead of learning the subject. Perfect on the past, useless on the future. A model with dozens of finely tuned parameters can fit historical noise so precisely it looks like genius, until it meets data it hasn't seen.
• Data snooping is testing fifty strategy variants and reporting the winner. With fifty random variants, the chance that at least one looks profitable by pure luck exceeds 92%. The vendor shows you the winner. The forty-nine losers don't get screenshots.
• Survivorship bias is backtesting on today's S&P 500 members only. Enron, WorldCom, Lehman Brothers vanish from the data, and the past looks rosier than it was. You're testing on the winners' league table.
• Look-ahead bias is accidentally using information that wasn't available at decision time, like triggering a trade on a bar's closing price using that same close. Invisible in code, fatal in production.

The red flags, as rules: win rates above 70–80%, profit factors above 3–4, Sharpe ratios above 3, strategies with no losing periods, models with ten-plus hand-tuned parameters. Professional quants target Sharpe ratios of 1–2. Good systematic equity strategies often run 0.5–1.0. If the numbers look too good to be true, they are measuring the backtest, not the strategy.

The only test that counts is out-of-sample data the strategy never touched during development. And there's a trap inside that trap: once you peek at out-of-sample results and tweak the model, that data isn't out-of-sample anymore. Professionals use walk-forward testing, fitting on a rolling window and testing the next window, repeated, to mimic real deployment. Amateurs optimize until the backtest shines. The backtest's job is to try to prove the strategy doesn't work. Adopt "my backtest is lying" as your default assumption, and you'll be right more often than the optimists.

Practitioner post-mortems suggest most machine-learning trading strategies don't survive six months in production, killed by overfitting, regime change, or costs the backtest undercounted. That's a practitioner estimate, not a peer-reviewed fact, but its direction rings true to everyone who's watched this closely: the market is very good at humbling models.

The planner's failure checklist

Before you trust any AI trading system, vendor product or your own code, demand answers to the failure questions:

1. What is the kill switch? One action that flattens positions and halts trading. If the answer is vague, walk away.
2. What is the daily loss limit? The system must halt itself at a predefined loss. Yours, not its.
3. What happens in a flash crash? Correlated selling, halted stocks, broken data feeds. Has anyone thought about it?
4. What are the position caps? No single position, no single day, should be able to end the account.
5. Show me the out-of-sample results, and tell me how many variants were tested before this one.

Knight Capital was a firm of professionals with every resource, and it nearly died in 45 minutes from a deployment error. Your vendor with the slick website and the "proprietary AI" has thought about failure less than Knight did. Price that in.

Next: the people who don't fail accidentally, the ones running AI trading as a deliberate fraud.