Max Headroom 
UNCLASSIFIED
If you want to make the most of The Field We Now Call AI, look to trading. Specifically, the tech-driven sort.
People who’ve read my other work, or who have had the misfortune of speaking with me one-on-one, have already heard this line. My long-running half-joke is that my AI consulting is based on best practices I picked up from trading way back when.
I say this with good reason. Modern trading—for brevity, I’ll lump algo(rithmic), electronic, quant(itative) finance, and any other form of Throwing Computers at the Stock Market under the umbrella of “algo trading”—applies data analysis and mathematical modeling to business pursuits. It’s full of hard-learned lessons that you can and should borrow for data work in other domains, even if your industry exists far afield of the financial markets. You can always ask, “How would algo trading handle this modeling issue/account for errors in this data pipeline/connect this analysis work to the business model?”
More recently I’ve been thinking about algo trading’s origin story. Which has led me to ask:
What can the computerization of Wall Street tell us about the rise of AI in other domains?
The short version is that the computers arrived and trading changed forever. But the truth is far more nuanced. Companies that internalize the deeper lessons from that story are poised to win out with AI—all of data science, ML/AI, and GenAI.
Let’s start with an abbreviated, slightly oversimplified history of technology in trading.
An Abbreviated History of the Delightful Chaos
At its core, trading is a simple matter of buy low, sell high: buy some shares of stock; wait for their price to go up; sell those shares; profit.
This is when you’ll point out that there are more complicated approaches which juggle shares from multiple companies…and that short-selling reverses the order to “sell high, buy low”…plus you have derivatives and all that… And I would agree with you. Those products and techniques certainly exist! But deep down, they are all expressions of “buy low, sell high.”
The mechanics of trading amount to strategy, matching, and execution:
Your trading strategy defines what shares you’ll buy, when to buy them, and when to sell. It can be as innumerate as “buy when the CEO wears black shoes, sell when they wear brown shoes.” It can involve deep industry research that tells you to move when the price exceeds some value X. Maybe you plot some charts to look for trends. Or you take that charting to the next level by building crazy mathematical models. However you devise your trading strategy, it’s all about the numbers: how many shares and at what price. You’re watching movements of share prices and you’re reacting to them, usually with great haste.
On the other side of strategy we have order matching and trade execution. Here’s where you pair up people who want to buy or sell, and then place those orders, respectively. In the olden days, matching and execution took place through “open outcry” or “pit” trading: people in a large, arena-like room (the pit) bought and sold shares through shouting (hence “outcry”) and hand signals (occasionally, the “catching hands” kind of signal). You watched prices on big screens and took orders by phone. Your location in the pit was key, as was your height in some cases, because you needed the right people to see you at the right time. Pit traders will tell you that it was loud and frenetic—like a sports match, except that every action involved money changing hands. Oh yes, and a lot of this was recorded on paper tickets. Messy handwriting and mishearing things led to corrections after-hours.
Computerization of these activities was a three-decade process—a slow start but a rousing finish. It began in the 1970s with early-day NASDAQ publishing prices electronically. (To drive the point home, note that the last two letters stand for “Automated Quotation.” You now have extra trivia for your next party conversation. You’re welcome.) Then came the UK’s 1986 “Big Bang” shift to electronic trading. Things really picked up in the 1990s through the early 2000s, which saw much wider-scale use of electronic quoting and orders. Then came decimalization and REG-NMS, which further encouraged computerized order matching and execution.
Combined, this led to a world in which you could get up-to-the minute share price data, find a counterparty with which to trade, and place orders—all without heading to (or calling someone in) the pit. Without hand signals. Without jumping up and down to be seen. Without the risk of fisticuffs.
From there, “pull in price data by computer” and “place orders by computer” logically progressed to “hire rocket scientists who’ll build models to determine trading strategy based on massive amounts of data.” And to top it off, remember that all of this electronic activity was taking place at, well, computer speeds.
Pit traders simply couldn’t keep up. And they were eventually pushed out. Open outcry trading is pretty much gone, and the role of “trader” has shifted to “person who builds or configures machines that operate in the financial markets.”
Understanding the Why
From a distance, it’s easy to write this off as “the computers showed up and the humans were gone. End of story.” Or even “the computers won simply because they were faster.” That’s the scenario AI-hopeful execs have in mind, but it’s far more complicated than that. It helps to understand why the bots took over.
I wrote a short take on this last year:
Trading is a world awash in numbers, analyses, and pattern-finding. In the pre-technology era, humans did this work just fine. But then computers arrived, doing the math better, faster, at a larger scale, and without catching a case of nerves. Code could react to market data changes so quickly that network bandwidth, not processor speed, became the limiting factor. In every aspect of the game—from parsing price data to analyzing correlations to placing orders—humans found themselves outpaced.
I’ll pause here to explain that trading happens in a marketplace. There are other participants, among whom there’s an element of competition (uncovering price shifts before anyone else and then moving the fastest on those discoveries) but also cooperation (as the person buying and the person selling both want to move quickly). That lent itself well to network effects, because once one group started using computers to parse market data and place orders, other groups wanted to join in and so they got their own. The traders who were still dealing in paper and hand signals weren’t so much competing with computers but with other traders who were using computers.
Continuing from that earlier write-up:
To understand what this meant for 1990s-era traders, imagine you’re a chess pro sitting down for a game. Except the board now extends to fifty dimensions and your opponent can make multiple moves without waiting for you to finish your turn. They react to your confused facial expression by explaining: the pieces could always do this; you just weren’t able to move them that way. That was the shift from open-outcry (“pit”) trading to the electronic variety. Human actors were displaced overnight. It just took them another few years to accept.
That sentence in bold gets to the core of why computerization was a runaway success. The desire for speed was always there. The desire for consistency under pressure was always there. The desire to find meaningful patterns in the mountains of pricing data was always there. We just couldn’t do that till computers came along. People figured out that computers could consistently, dispassionately multitask on market matters while crunching massive amounts of data.
From that perspective, computers didn’t really take human jobs—humans were doing jobs that were meant for computers, before computers were available.
Computers and trading made for a perfect marriage.
Well, almost.
It’s Not All Roses
All of these computers jockeying for position, operating at machine speeds, introduced new opportunities but also new risk exposures. New problems cropped up, notable for both their magnitude and ubiquity: high-speed cheating, like order spoofing; flash crashes; bots going out of control… Traders and exchanges alike implemented new testing and safety procedures—layers upon layers of risk management practices—as a matter of survival. It was the only way to reap the rewards of using bots while closing off sources of ruin.
Tech-related incidents still happen, like the 2012 Knight Capital meltdown. And bad actors still get away with things now and then. But when you consider the size and scale of the model-driven, electronically traded financial markets, the problems are relatively few. Especially since every incident is taken as a learning experience, leading traders and exchanges to institute new policies that discourage similar problems from cropping up down the road.
Frankly, the most notorious incidents in finance—like the 2008 mortgage crisis or the self-destruction of hedge fund LTCM—were rooted not in technology but in human nature: greed, hubris, and people choosing to oversimplify or misinterpret risk metrics like VaR. The computerization of trading has mostly been positive.
Learning from the Lessons
That trip through trading history brings us right back to where I started this piece:
If you want to make the most of The Field We Now Call AI, look to trading. Specifically, the tech-driven sort.
The move from the pits to computerized trading holds lessons for today’s world of AI. If you’re an executive who dreams of replacing human headcount with AI bots, you’d do well to consider the following:
Give the machines machine jobs. Notice how traders and exchanges applied computers to the work that was amenable to automation—matching, execution, market data, all that. The same holds for AI. That manual task may annoy you, but if AI isn’t capable of handling it just yet, it must remain a manual task.
Machines give you “faster”; you still need to figure out “better.” Does the AI solution provide an appreciable improvement over the manual approach? You’ll need to run tests—the kind where there is an objective, observable, independently verifiable definition of success—to figure this out. Importantly, you’ll need to run these tests before modifying your org chart.
The machines’ speed will multiply the number and scale of any errors. This includes the error of using AI where it’s a poor fit. Avoid doing the wrong thing, just faster.
This is of special concern in light of the wider adoption of AI-on-AI interactions, such as agents. One bot going out of control is bad enough. Multiple bots going out of control, while interacting with each other, can lead to a meltdown.
Technology still requires human experience. While bots have taken over the moment-to-moment stock market action, they’re built by teams of experts. The computers are useless unless backed up by your team’s collective domain knowledge, expertise, and safety practices.
Tune your risk/reward trade-off. Yes, you’ll want to develop controls and safeguards to protect yourself from the machines going off the rails. And you’ll need to think about this at every stage of the project, from conception to R&D to deployment and beyond. Yes.
Yes, and, you’ll want to think beyond your downside exposures to consider your upside gain. Well-placed AI can bring about massive returns on investment for your company. But only if you choose the AI projects for which the risk/reward trade-off plays in your favor.
You’re only in competition with yourself. Traders try to get ahead of each other, to detect price movements and place their orders before anyone else. And they place trades with one another, each taking a different side of the same bet (and hunting for counterparties who will make bad bets). But in the end, as a trader, you’re only in competition with yourself: “How did I do today, compared to yesterday? How do I avoid mishaps today, so I can do this again tomorrow?”
The same holds for your use of AI. Executives are under pressure—whether from their investors, their board, or simple FOMO as they read about what other companies are doing—to apply AI anywhere, everywhere. It’s best to look inside and figure out what AI can do for you, instead of trying to copycat the competition or using AI for AI’s sake.
What if…?
I opened with a question about algo trading, so it’s fitting that I close on one. To set the stage:
In the early days of data science—a good 15 years before GenAI came around—I hypothesized that traders and quants would do well in this field. It was a smaller and calmer version of what they were already doing, and they had internalized all kinds of best practices from their higher-stakes environment. “If Wall Street pay ever sinks low enough that those people leave,” I mused, “the data field will definitely change.”
Wall Street comp never sank far enough for that to happen. Which is good for the folks who still work in that field. But it also means I never got to thoroughly test my hypothesis. I still wonder, though:
What if more people with algo trading experience had entered the data science field early, and had spread their influence?
Imagine if, in the early to mid-2010s, a good portion of corporate data departments were built and staffed by former traders, quants, and similar finance professionals. Would we still see the meteoric rise of GenAI? Would companies be just as excited to throw AI at every possible problem? Or would we see a smaller, more focused, more effective use of data analysis in the pursuit of profit?
In the most likely alternate reality, the companies that genuinely need AI are doing well at it. Those that would have passed up on AI in our timeline come much closer to reaching their full AI potential here. In both cases the data team is deeply connected to, and focused on, the business mission. They adhere to metrics that allow them to track model performance. To that point, the use of those AI models is based on what those systems are capable of doing rather than what someone wishes they could do.
Importantly, these quant-run shops exhibit a stronger appreciation of risk-taking and risk management. I use those terms in the finance sense, which involves fine-tuning one’s risk/reward trade-off. You don’t just close off the downsides of using automated decision making; you aggressively pursue additional opportunities for upside gain. That involves rigorous testing during the R&D phase, plus plenty of human oversight once the models are running in production. It’s very much a matter of discipline. (Compare that to our timeline, in which the Move Fast and Break Things mindset has bolstered the Just Go Ahead and Do It approach.)
Interestingly enough, this alternate timeline still sports plenty of companies that use solely AI for the cool factor. There are just no quants or traders in those AI departments. Those people are finely attuned to using data in service of the business goal, so a frivolous use of AI sends them running for the exit. If they even join the company in the first place.
All in all, the companies in the alternate timeline that need AI are doing quite well. Those that don’t need AI, they’re still making the snake oil vendors very happy.
Today’s GenAI hype machine would certainly disagree with me. But I’ll point out that the GenAI hype doesn’t hold a candle to the tangible, widespread impact of the computerization of trading.
Food for thought.
Artificial Intelligence, Commentary
Radar
