It's easier to forecast volatility and correlations than returns

There are three things we would ideally like to have when constructing portfolios. The most obvious is an expected set of returns for each asset. This is what most people spend their time worrying about and trying to estimate.

But we’d also like to have an estimate of volatility so that we can understand how certain or uncertain the returns may be. Finally, we want an estimate of correlations so that we can combine assets to reduce volatility.

Assuming we could forecast returns, volatility, and correlations, portfolio construction would just be a math problem. But how hard is forecasting each of these three things? And how do we incorporate varying forecastability into answering the big questions of asset allocation and diversification?

We looked at four daily return datasets with long histories: equities, Treasurys, corporate debt, and commodities. The returns are in excess of the risk-free rate, as this is what is fed into portfolio optimizations. Beginning in 1989, we divided the data into 62-day periods (trading days in a quarter) and created 139 independent quarterly observations of returns, volatilities and correlations.

We first looked at how much returns, volatility, and correlations move around. The applicable statistical concept here is the standard deviation, which is a measure of the average distance of each observation from the mean. The higher the standard deviation, the higher the uncertainty and the more benefit there is to having predictions that lower that uncertainty. As a reminder, if the series is normally distributed around its mean, then 68% of the observations will be within +/- one standard deviation of the mean. Below we show the mean 62-day return and standard deviation for each of the return series.

Figure 1: Mean 62-Day Return and Standard Deviation

Source: Bloomberg. Equity returns are S&P 500 Total Return index, Treasury returns are Bloomberg US Intermediate Treasury index, Commodity returns are S&P GSCI Total Return index, and Corporate returns are Bloomberg US Corporate Total Return index.

Unsurprisingly, returns move around a lot. We’d really like to have a better estimate of returns than just the mean.

What about volatility and correlations? How good an estimate is the historical average for those series?

Figure 2: Mean 62-Day Return and Standard Deviation for Returns, Volatility, and Correlations.

Source: Bloomberg, Verdad analysis.

Volatility, as it turns out, is not particularly volatile. Guessing the mean volatility for each of these series is going to be a lot closer to the realized volatility than guessing the mean return would be to the realized return.

Correlations look more like returns. The standard deviations of the correlations are high relative to their means—and the correlations often swing from positive to negative.

We then tested a simple predictive model for returns, volatility, and correlations, relying only on the previous period value and the historical average value (up until the point of measurement). This is not a sophisticated approach, but it’s a reasonable way to understand whether each series trends (meaning the previous period value will be important), mean reverts (meaning the running average will be important), or follows a random walk (meaning historic data has no predictive ability).

This simple approach produces encouraging results for correlations and volatilities, but not for returns (a result that shouldn’t be surprising to anyone familiar with the Efficient Markets Hypothesis). Below we show the R squared for the regressions.

Figure 3: R2 for Predictive Return, Volatility, and Correlation Regressions

Source: Bloomberg, Verdad Analysis.

Although returns appear to follow a random walk, we see that this simple model actually does an okay job of predicting volatility and a good job at predicting some of the correlations. Another way to assess the quality of these models is to compare the root mean squared error (RMSE) from our regressions to the standard deviation. Whereas standard deviation measured how much the series deviated from the mean, RMSE is calculated the same way, except it measures how far the series deviates from the regression estimate. An RMSE lower than the standard deviation indicates that we can reduces our range of outcomes versus just using the mean.

Figure 4: Regression RMSE vs Standard Deviation

Source: Bloomberg, Verdad Analysis.

Our simple regressions do not reduce our range of outcomes for returns at all, and they yield only a slight improvement for volatility. But for correlations they have a very large impact. Our range of outcomes has been reduced by +/- 11 correlation points for the equity Treasury correlation. This is very promising. Why does it work so well for correlations? We can tell by looking at the statistical significance of the coefficients from the regression (a p-value of 5% or below is significant).

Figure 5: P-Values for Regression Coefficients

Source: Verdad Analysis

Correlations are predictable because they both trend (significant p-value for the previous value) and mean revert (significant p-value for the running average). Volatility is somewhat predictable because it trends.

What does this all mean? It means that when we develop our forward estimates of returns, volatilities, and correlations for our asset allocation models, we must be very humble about our asset class return predictions (as distinct from relative returns within an asset class where we believe factor models work). Our estimates are likely to have a high degree of uncertainty and error. By contrast, we can be bolder in our estimates of volatility because volatility tends to be more range bound and our mistakes are less likely to be extreme. Finally, we should be bold with our correlation predictions because there is both room for improvement over using the mean and correlations appear to be more predictable. Investing our time in building good correlation prediction models has the potential to yield the most benefit.

Acknowledgment: Our interns Isabella Goudros and Ben Rosa worked on this research. Isabella is a rising senior at Harvard studying economics and global health policy. She is open to exploring many career paths after college but is particularly drawn to consulting and foundation investing. Beyond the classroom, Isabella is a heptathlete on Harvard's track and field team, looking to represent Canada at the 2028 Olympics. Ben is also a rising senior at Harvard. He studies economics and is interested in pursuing a career with the IRS after college. Beyond the classroom, Ben is on the track and cross country teams. Please reach out to us if you would like to connect with Isabella or Ben.

back to library

Archive

Forecastability and Portfolio Optimization