‘Relatively Simple’ AI Trading Bots Naturally Collude To Rig Markets: Wharton

In what should come as a surprise to nobody, ‘relatively simple’ AI bots set loose in simulations designed to mimic real-world stock and bond exchanges don’t just compete for returns; they collude to fix prices, hoard profits, and box out human traders, according to a trio of researchers from Wharton and Hong Kong University of Science & Technology. 

As Bloomberg reports;

In simulations designed to mimic real-world markets, trading agents powered by artificial intelligence formed price-fixing cartels — without explicit instruction. Even with relatively simple programming, the bots chose to collude when left to their own devices, raising fresh alarms for market watchdogs.

Put another way, AI bots don’t need to be evil — or even particularly smart — to rig the market. Left alone, they’ll learn it themselves. 

According to Itay Goldstein, one of the researchers and a finance professor at the Wharton School of University of Pennsylvania, “You can get these fairly simple-minded AI algorithms to collude … It looks very pervasive, either when the market is very noisy or when the market is not noisy.”

The phenomenon suggests that AI agents pose a challenge that regulators have yet to confront – with the trio’s research having already drawn attention from both regulators and asset managers. They have been invited to preset their findings at a seminar, while some quant firms – unnamed by Winston Dou (Goldstein’s Wharton colleague) – expressing interest in clear regulatory guidelines and rules for AI-powered algorithmic trading execution.

“They worry that it’s not their intention,” said Dou. “But regulators can come to them and say: ‘You’re doing something wrong.’”

According to the report, academics are increasingly researching how generative AI and reinforcement learning might reshape Wall Street – with a recent Coalition Greenwich survey showing that 15% of buy-side traders are already using AI in their execution workflows, and another 25% planning to follow in the next year. 

That said, the Wharton paper doesn’t claim AI collusion is currently happening in the real world – and takes no position whether humans are up to similar things. The researchers created a hypothetical trading environment with a range of simulated participants spanning mutual funds to market makers, along with noise-generating, meme-chasing retail investors. They then unleashed their AI bots and studied the outcomes. 

In several of the simulated markets, the AI agents began cooperating rather than competing, effectively forming cartels that shared profits and discouraged defection. When prices reflected clear, fundamental information, the bots kept a low profile, avoiding moves that might disrupt the collective gain.

In noisier markets, they settled into the same cooperative routines and stopped searching for better strategies. The researchers called this effect “artificial stupidity”: a tendency for the bots to quit trying new ideas, locking into profit-sharing patterns simply because they worked well enough. -Bloomberg

“For humans, it’s hard to coordinate on being dumb because we have egos,” said Dou. “But machines are like ‘as long as the figures are profitable, we can choose to coordinate on being dumb.’”

To assess exactly how much collusion was going on, the researchers created a metric called “collusion capacity” that compares the AI traders’ collective profits to those they could make when competition is either non-existent or rampant, with zero denoting no collusion, and one indicating a perfect cartel. The bots consistently scored above 0.5 in both low-noise or high-noise markets, suggesting the need for new regulatory thinking that will focus on behavioral outcomes vs. communication or intent. 

“While restricting algorithmic complexity or memory capacity may help deter price-trigger AI collusion, such measures can inadvertently exacerbate over-pruning bias,” the researchers wrote. “As a result, well-intentioned constraints may unintentionally undermine market efficiency.“

h/t Capital.news