Man + Machine

Our investment process is a hybrid of man and machine.  We use the reference class forecasting methodology championed by Daniel Kahneman and Philip Tetlock.  We start with quantitative, base rate–driven analysis to focus our attention on a pool of cheap, leveraged companies that we think are unlikely to go bankrupt. We then analyze each individual company and form our portfolio based on our own human judgment.

Like the legislator turned Supreme Court justice, Joseph Story, we play a dual role. We study and create the laws that govern our investment process, but we also judge whether a particular security complies with the laws we have adopted.

Our combination of quantitative and qualitative analysis is unusual: most firms that do one despise the other. We take a more balanced approach, following the methodology of Daniel Kahneman and the spirit of Deng Xiaoping, who remarked: “It doesn't matter whether a cat is white or black, as long as it catches mice.” (Loyal readers be assured, Deng is the only Communist we will ever quote approvingly.)

For this week’s note, we have conducted a thorough review of our last three years of investing to assess how well we’ve done at playing this dual role: How well have our quantitative tools performed? Have we added value through our fundamental analysis and portfolio construction relative to the raw quantitative output?

In the table below, we show the performance of the market index, the performance of our quantitative screens, and the performance of our global fund (read on for a description of our methodology in running this analysis).

Figure 1: Man + Machine

Source: Verdad, CapitalIQ

Over the past three years for the total fund, the quantitative factors added 6% while our qualitative stock selection had an estimated -1% impact. However, this was largely the result of 2015, in which we were more heavily weighted in the US than the screen and the US dramatically underperformed the rest of the world likely because of the shale drilling–spurred high-yield debt crisis. Over the past two years, the quantitative screen added 15%, while our qualitative stock selection added an additional 6%. In 2017, our qualitative judgment added 5% whereas the quantitative factors added 5%. Over the three years, the fund generated significant alpha.

At a high level, both cats seem to hunt mice. And our decision to use both man and machine to analyze and choose securities looks to be the right one. But we wanted to look a level deeper, to segment our portfolio by geography and compare the performance of our quantitative and qualitative processes in each region over each of the past three years to get a more granular sense of how often our processes have been effective in alpha generation.

Below we show the results of this analysis split by region. In the first column, we shade green if the quantitative screen beat the passive index. In the second column, we shade green if our actual picks outperformed the screen. And in the third column, we shade green if Verdad’s portfolio beat the passive index.

Figure 2: Man + Machine by Region by Year

Source: Verdad, CapitalIQ

In our analysis, our screen beat the market in 56% of region-years, our qualitative picks outperformed the screen in 67% of region-years, and our process combining qualitative and quantitative analysis produced returns that beat the market in 78% of region-years.

We are happy that these results suggest that both the man and the machine elements of our process seem to work the majority of the time—and seem to be additive and complementary to each other.

This makes sense to us philosophically. Our quantitative research allows us to build laws that bound our judgment. We believe these rules are essential to a good decision-making process. “The advantage of normally proceeding through the mediation of rules is enormous. It enables a person to consider and form an opinion on the general aspects of recurrent situations in advance of their occurrence,” noted the great twentieth-century philosopher of jurisprudence Joseph Raz. ”It enables a person to achieve results which can be attained only through an advance commitment to a whole series of actions, rather than by case-by-case examination.”

However, like a judge, we recognize the ambiguity under the law embedded in individual cases. And we believe that our rules, like the common law, should evolve through experience and precedent to take into account new information and complexities that no law could anticipate.

As we study the companies that comply with the letter of our quantitative laws, we not infrequently discover aggravating circumstantial evidence: one company might have a massive unfunded pension liability, another might be under SEC investigation, another might just have lost a major customer that isn’t reflected in the historic financials. Less frequently, we discover mitigating circumstantial evidence: one company might be selling parts to a cyclical industry that is at all-time lows, another might be awaiting a court judgment on a lawsuit that could bring them hundreds of millions in damages, yet another might have recently finished paying off a massive environmental liability. Our clinical judgement between firms that conform to the letter of our laws and, after diligence, appear to conform to the spirit of the laws seem likely, both by logic and by evidence, to have add additional value above and beyond a purely passive application of the letter of the law.

And so, at the end of the day, we are pragmatists like Deng. And we take Deng’s advice to “cross the river by feeling for stones,” trying to do what’s best while acknowledging the uncertainty and difficulty of the task. And in keeping with that philosophy, we plan to follow in the footsteps of Story, Raz and others, carefully and diligently applying a fixed law while acknowledging the humanity and uniqueness of individual situations.

Methodology
We took our screens from 12/31/2014, 12/31/2015, and 12/31/2016 and built portfolios of the top 50 names on those screens (eliminating any with below $200K of daily trading volume). The screens did not use the same methodologies. The initial screen ranked based on value and leverage primarily, with no consideration for quality. A year later, we had added quality screens and were using the version of the screen developed in our paper, “Leveraged Small Value Equities.” The following year, we added short interest to our screener. We added a 2% fee and expense drag to these portfolios to make them apples to apples with our live track record. We compared the performance of these portfolios to our live track record.

We also wanted to examine how we did in different regions. We divided the 50 stock portfolios into three regions (North America, Europe and Asia) to allow for easy comparison. We then took our live portfolios at the beginning of each year and divided these by region. We compared the performance of these beginning-of-year live portfolios to the regional quantitative portfolios. These regional portfolios do not account for inter-year trades.

We compared these portfolios to passive Vanguard indices: the Vanguard Total World Index for the global fund, the Vanguard Total Stock Market Index for North America, the Vanguard Europe Index for Europe, and the Vanguard Asia-Pacific Index for Asia.  The Vanguard Total World Stock Index Fund seeks to track the performance of a benchmark index that measures the investment return of global stocks. Seeks to track the performance of the FTSE Global All Cap Index. Broad exposure across developed and emerging equity markets around the world, including the United States. Vanguard Total Stock Market Index Fund is designed to provide investors with exposure to the entire U.S. equity market, including small-, mid-, and large-cap growth and value stocks.  The Vanguard Europe Index seeks to track the performance of the FTSE Developed Europe All CapIndex, which measures the investment return of stocks issued by companies located in the major markets of Europe. The Vanguard Asia-Pacific Index seeks to track the performance of the FTSE Developed Asia Pacific All Cap Index, which measures the investment return of stocks issued by companies located in the major markets of the Pacific region.