The Complicated Future of AI in Financial Markets

It is often said that AI — artificial intelligence — is the future. Technology is the source of soaring hope and pronounced concern, from smartphones to smart homes, appliances, robotics, and more. Data-driven with seemingly boundless applications, though, it’s probably here to stay. But will we see AI in financial markets?

The quantitative techniques used in AI are nothing new to financial markets. They were developed years ago with limited applications. Two developments, however, fueled an explosion of interest in AI: the collection of vast volumes of data and the development of cheap computing power to crunch numbers.

In the context of finance, those techniques evolved into several innovations. Systems following set rules could automate laborious calculations and signals. So traders and fund managers could accurately parse more information than ever before to direct their strategies. Eventually, these systems were able to automate complex rules-based trading.

But as AI technology unlocks new opportunities, dreamers wonder what might come next. Just as importantly, they ask, “Should we walk those roads?”

AI Capabilities are Expanding

During the summer of 2021, AlphaFold made a significant breakthrough. Billed by Forbes as “the most important achievement in AI — ever,” AlphaFold announced an artificial intelligence program that predicts the structures of known proteins.

Why does that matter? As the same Forbes article explained:

Proteins are involved in basically every important activity that happens inside every living thing, yourself included: digesting food, enabling muscles to contract, moving oxygen throughout the body, attacking foreign viruses and bacteria. Your hormones are made out of proteins; so is your hair.

[…]

But proteins do not stay one-dimensional. In order to become functional, they first fold into complex three-dimensional shapes.

A protein’s shape relates closely to its function. To take one example, antibody proteins fold into shapes that enable them to precisely identify and target particular foreign bodies, like a key fitting into a lock. Understanding the shape that proteins will fold into is thus essential to understanding how organisms function and ultimately how life itself works.

This makes AlphaFold’s functionality more than relevant, but its most significant contribution was a function of speed and intelligence. Before applying AI to the problem, it took about three years to understand the structure of a single protein. AlphaFold’s achievement in about one year is mind-blowing. It moves us past understanding a static protein structure and into a future of understanding how, where, and when that structure likely works.

That kind of power is, understandably, attractive to investors and fund managers hoping to stay ahead of the curve. It’s particularly interesting to consider AI techniques for risk management and for developing profitable trading models. Indeed, market participants currently use some AI on time-series data to work toward profitable trading strategies. But as AI progresses, what does AI’s future look like in financial markets?

The Difficulty in Applying AI in Financial Markets

The biggest roadblock to AI applications in financial markets is the nature of what the technology is trying to predict. Financial markets are complex adaptive systems (CAS). The adaptive nature means that participant behavior changes to adapt to other participants’ behaviors.

Adaptivity means that historical data from which AI algorithms attempt to extract tradable predictions don’t represent current market dynamics. A successful trading strategy may last for the shortest amount of time before that strategy causes adaptations. At that point, the strategy might no longer work, so you have an arms race among the market participants. They will use these previously successful strategies to extract profits from the market before others discover them and apply them. Then the same strategies are no longer effective.

In other words, there’s truth in the phrase “past performance is not indicative of future results.”

Consider Goodhart’s law. Named after economist Charles Goodhart, the principle goes back to one of his 1975 writings:

Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.

In other words, the correlation between two variables we notice today might not be there tomorrow. The statistical correlation breaks down because targeting causes people’s behavior to change. That causes the previously observed relationship to break down.

Financial Market AI Complications in a World of HFT (High-Frequency Trading)

Another significant concern when discussing the applicability of AI in financial markets is that quantitative market navigation is complicated by the speed at which we trade. Computers and algorithms have enabled us to execute trades at a rate most of us cannot comprehend. Technology has given rise to the dominance of what is known as HFT. Vox explains it this way:

If you’re an average human being, your eyes take around 400 milliseconds to blink once. High-frequency trading is a kind of market activity that moves in less than one millisecond to spot and take advantage of an opportunity to buy or sell. It happens through trading algorithms, programs that determine how to trade based on fast-moving market data.

Traders who use this strategy are not sitting at home clicking away in their E*Trade account. As Michael Lewis explained in his book Flash Boys, entire operations have relocated to be just a little bit closer to the exchanges. They, therefore, get their information just a few nanoseconds earlier because the data didn’t travel as far to their systems. They aren’t looking to make a few nickels and dimes on their trades. They’re targeting fractions of pennies at very high volume.

When we say high volume, we mean it. It is estimated that HFT constitutes half of all volume in U.S. stock market trading. That’s where AI in financial markets starts to get tricky.

If behavioral economics and statistical analyses tell us the market will adapt to highly-sophisticated trading algorithms and half of all trading is guided by such algorithms and it’s happening at breathtaking speeds, what’s at risk?

As it turns out, we’ve already gotten a glimpse of that. And no, it wasn’t 2008.

Crash Course on the AI-Aided Flash Crash

On May 6th, 2010, a trader in Kansas City placed a large trade in the E-mini futures market, or S&P 500 futures contracts. It was out of step with ongoing market conditions and set off alarm bells for algorithms around the world. The result was nearly unbelievable. As a Hastings Business Law Journal article recounts:

Twenty minutes later, the broad U.S. securities markets were down almost a trillion dollars, losing at their lowest point more than nine percent of their value. Certain stocks lost nearly all of their value from just minutes before. Faced with the blistering pace of the decline, many market participants opted to cease trading entirely, including both human traders and High-Frequency Trading (“HFT”) programs. This withdrawal of liquidity accelerated the crash, as fewer buyers could absorb the rapid-fire selling pressure of the HFT programs. Within two hours, prices were back near their pre-crash levels. It took the SEC more than five months of research to determine what sparked the crash.

This precipitous market crash was made possible, if not inevitable, by the growing interconnectivity of the securities markets and the proliferation of high-frequency trading. These two factors combine to create a situation where trillions of dollars of wealth can be destroyed before human traders can react, often for reasons that are unknown even to the program’s designers. As a result, extreme market movements can perpetuate themselves across the U.S. markets almost entirely without human involvement.

It was a significant moment in trading history. That event prompted fund managers and traders to establish new programmatic measures to prevent getting caught up in such a storm in the future. But if Goodhart’s law holds merit, is there really any adequate way to prepare for such events?

Now add AI into the mix. We may be moving beyond algorithms based on models using prior market data and into a realm where technology interprets current quantitative inputs to predict what comes next. Isn’t the technology still vulnerable to the same pitfalls? Do we expect a computer to fully process geopolitical events or headline-driven behavior?

Or is this just us asking for more pain while we figure out inevitable glitches?

Why Sustainable AI in Financial Markets is Unlikely

It may be tempting to look at AlphaFold’s achievements and think we’re nearer to advanced AI applications in finance than we think. After all, if we can get closer to understanding the nuances of human biology, and life itself, in that short timeframe, surely we can get there in a world of numbers.

The thing to remember is that biology moves slowly. We have time to apply information gleaned from studies of biological data. An economic scandal or geopolitical crisis will not influence how a protein folds.

It doesn’t matter how close traders get to those exchanges, or how fast technology allows us to move. Predictive technology is more likely than not to fail, and it may cause significant damage. Anyone seeking their fortune through AI in financial markets is likely to be disappointed.

[Editors’ Note: To learn more about this and related topics, you may want to attend the following on-demand webinars (which you can listen to at your leisure and each includes a comprehensive customer PowerPoint about the topic):

• Data Privacy Compliance
• Best Practices Regarding Technology]

©2022. DailyDAC^TM, LLC d/b/a/ Financial Poise^TM. This article is subject to the disclaimers found here.