Making Sense of the Machine Learning Arms Race

There is a running joke in Silicon Valley that if you want to sell your startup for a billion-dollar valuation, be sure to include the buzzwords “machine learning,” “AI,” and “automation” in your pitch book. Not to be outdone, marketers in the financial industry have also caught on to the wave. We’re beginning to see a proliferation of investment products that seek to apply “proprietary technology” and use “alternative data” to gain an informational edge in the market.

To be fair, advances in machine learning do offer real benefits to society by enabling systematic decision-making at scale. By letting the machines handle the repetitive task of churning vast amounts of information into mechanical predictions, humans can focus on jobs that require creativity and context. Machine learning can also complement human judgement in an investment process, as we have written about before.

In this week’s article, we will abstract away from the mechanics of machine learning in order to focus on universal principles for evaluating these algorithms in an investment strategy. The key question for investors is how to separate rhetoric from substance when evaluating an investment strategy that uses machine learning. We believe the key differentiator is whether a strategy seeks to generate alpha through an informational edge or from compensation for taking on sensible factor exposures.

Where’s the Beef?

The distinction between an informational edge and factor exposures is important because an informational edge will always be competed away when other people learn about it. Hence, the fragility of informational strategies creates a strong incentive for managers to label them as “proprietary” secrets.

On the other hand, the most powerful and well-documented factor exposure is value. Despite widespread knowledge of this factor since at least the days of Benjamin Graham, it remains persistent over the long haul. And the value premium will remain persistent as long as most people are unwilling to bet against the consensus. Contrarians earn a return premium from buying securities that are cheap because of consensus pessimism. And this reward is compensation for bearing the risk that the consensus may occasionally be right when a security deserves to be cheap. In other words, contrarians make money by bearing the risk of being wrong. Over the next couple of weeks, we will elaborate on this concept and demonstrate how the pursuit of higher expected returns necessarily involves a higher probability of being wrong. We will also show how machine learning can mitigate the risk of being wrong by fine-tuning the forecasts from linear models.

For now, we simply propose that any alpha from an informational edge is bound to be tenuous over the long term. Many of the machine learning approaches we read about involve pairing alternative data sets with machine learning to gain an informational edge. Investment funds are spending vast amounts of money purchasing and processing alternative data, with industry analysts saying it costs a minimum of $1.5 million per year to even start taking advantage of alternative data. These firms are buying satellite imagery of parking lots and shipping tankers to predict retail sales or developing machine learning algorithms to scan raw social media data and read earnings call transcripts.

We are skeptical that these informational approaches are useful for making investment decisions.

Even if information is derived from proprietary datasets—for example, a vendor might sell you exclusive access to car insurance data so you can predict Ford’s sales next quarter—the key question is whether the return benefit from using this private information exceeds the cost of purchasing it. To evaluate this question empirically, we can look at the performance of US active managers, since they invest significant resources into obtaining information about individual stocks.

Costly Information

In their 2009 paper, Eugene Fama and Ken French evaluated 22 years of US mutual fund performance, net of fees. This is a useful study for our purposes because fees represent the cost of acquiring and processing information. After all, most active managers buy research reports from Wall Street banks, hire the best analysts from top business schools, and fly these analysts to meet with managers of portfolio companies. How much profit has been created for investors from these massive research budgets?

Fama and French found that at least 67% of active managers underperform their benchmark after fees. This means more than two-thirds of active managers fail to convert their costly information into a profit for investors. This is worse than a random result. If the average active manager has zero informational edge net of fees, only half of them would underperform their benchmark by chance—just like a coin flip.

Spending more money does not appear to deliver better quality information, either. The figure below shows the proportion of US equity mutual funds that outperform (winners) and underperform (losers) their benchmark over a ten-year horizon, categorized by quartile of expense ratio. Funds with higher fees—and thus, higher information acquisition costs—are more likely to underperform their benchmark.

Figure 1: Proportion of Winners and Losers by Quartile of Expense Ratio (10-Year Horizon)

Source: Dimensional Fund Advisors, “Mutual Fund Landscape” (2018)

Returns Are Noisier Over the Short Term

The second challenge for informational strategies is time. Since most sources of alternative data were created within the past 15 years, there is a shorter history of alternative data relative to traditional datasets. For example, the University of Chicago’s CRSP dataset has US stock prices since 1926 and quarterly fundamentals since 1963. This means traditional factor models can be trained on over 50 years of data, whereas machine learning models that rely on alternative data must be trained on less than 15 years of data.

Figure 2: Alternative Data Have a Much Shorter History than Traditional Data

Source: Verdad research

Since any information gleaned from alternative datasets has a short shelf life before it is broadly known to the market, informational strategies generally predict over short time horizons. Perhaps training a machine learning algorithm on daily stock prices and Twitter data might turn up some relationships in a backtest. But the reliability of these signals in real life is likely to be very low. This point is illustrated in Figure 3 below, which shows the average equity premium (excess return of stocks relative to short-term bonds) over various time horizons. The Sharpe Ratio, which is analogous to a signal-to-noise ratio, is close to zero over daily horizons and increases over longer horizons.

Figure 3: US Equity Premium is More Predictable Over Longer Horizons (Jan 1963 to Dec 2018)

Source: Ken French data library and Verdad research

Evidently, the larger role of noise in short-term price movements is problematic for machine learning algorithms that attempt to predict short-horizon returns.

Long-Horizon Returns Are Compensation for Factor Exposures

The most reliable investment signals are related to sensible factor exposures. By virtue of being economically sensible, the best investment signals can forecast returns over long horizons. A good example of this is the contrarian value factor, which provides stable return forecasts over long horizons in exchange for bearing the risk of betting against the consensus.

Figure 4: Annualized US Equity Returns by Quintile of Valuation (Jan 1963 to Dec 2018)

Source: Ken French data library and Verdad research

What does this mean for machine learning? First, the role of machine learning should be incremental to traditional factor models that sort stocks by factor exposures like value. Just because machine learning is now available as a new tool doesn’t mean that good old linear regressions should be thrown out.

Second, if your investment strategy pursues higher expected returns, then the machine learning overlay should still leave you with a basket of unloved securities. Put simply, if your strategy targets a 20% expected return, then the machine learning overlay should leave you with a portfolio that generates a free cash flow yield of around 20%. This applies even if you are using machine learning to manage your risk of being wrong by separating the good 20% free cash flow yield companies from the bad ones.

For example, at Verdad we use machine learning to identify companies that are more likely to pay down debt. But at the end of the day, the companies that are most likely to pay down debt also have the highest amount of leverage and trade at the lowest valuations.

Figure 5: Out-of-Sample Forecasts of Debt Paydown in Europe (Jun 1997 to Dec 2017)

Sources: S&P Capital IQ and Verdad research

A traditional factor model might find that cheaper and more leveraged stocks perform better, but the machine learning algorithm shows us that a key reason for this outperformance is that these stocks are much more likely to pay down debt—which then drives equity returns. And the machine learning algorithm is better able to predict deleveraging by integrating a wider array of variables than the simple combination of leverage and value.

Traditional factor models have already identified the most powerful predictors of future equity returns. However, this does not mean that linear models are perfect. It simply means that machine learning should be used to complement traditional models that sort stocks by factor exposures like value. This makes logical sense because progress in any scientific field is usually incremental.

Over the next two weeks, we will explore how we use machine learning to fine-tune forecasts from linear models. And we will discuss one of our most intriguing research findings about why these models work—one that provides important new insights into market efficiency and what makes sensible factor strategies likely to persist over time. We look forward to updating you on the significant progress we have made in using machine learning to improve on traditional factor models—and what machine learning can reveal about how equity markets work.