Wednesday, March 18, 2015

Past in Not Always Prologue -- a rebuttal of “It’s Not the Heat, It’s The Tepidity” by Clifford Asness and Aaron Brown

Clifford Asness is a hedge fund manager and co-founder of AQR Capital Management, with degrees in English, Economics and Finance.  Together with his colleague Aaron Brown, he recently published a note that analyzes global warming data and concludes that the data do not support the projections made by climate-science based IPCC models.  In these comments, I will point out some of the problems with the analysis of Asness and Brown (hereafter referred to as AB), which make their conclusions somewhat unsubstantiated and perhaps even misleading.

AB start with the global land-ocean annual mean surface temperatures, as shown in their Figure 1.  This is their “data”, where time is the independent variable or “predictor”, and the global temperature index (i.e., deviation of the recorded temperature from the 1951-1980 average) is the dependent variable or “response”.  They proceed to fit a linear regression trend line to the data (Figure 2), and compare the projections from this simple “model” with IPCC model results (Figures 3-5).  Since the linear fit in Figure 2 does not fully match the curvilinear trend in the data, especially for the post 1950 period, AB then fit a quadratic regression model to the data and compare the corresponding updated projections to IPCC model results (Figure 6).  Their main conclusion is that the global warming trend, extrapolated from a time series of past temperature readings, is at odds with IPCC projections.  To quote AB, “You can believe the models if you like, or you can look at the data and assume the most likely future is an extrapolation of the past (my bold).  What you cannot do is both.”

Let us consider the following thought experiment.  I want to predict the future movement of the S&P 500 index.  Using AB’s analytical premise, I should be able to look at the past history of this index and project it into the future.  To illustrate this point, I will use a time series of the S&P 500 Index values from 1950 until the present.  The figure below shows a chart of this data, where the index values are plotted in a logarithmic scale per normal practice.  Next, I fit a log-linear model to this data, with an R^2 value of 0.9533, indicating an excellent fit. 



Even though this fit is as good as what AB show with their analysis, does this give me any confidence that I can predict the future trajectory of the S&P 500 based on past history?  Not really.  Would any investment manager make decisions based on these projections of the S&P 500’s values – without doing his or her due diligence about future conditions?  I think not.  A reasonable analysis would require us to consider how future estimates of corporate earnings, interest rates, Eurozone conditions, investor sentiment, etc., compare to the immediate past and then decide if inferences about the future can and should be made from past data.

Extending this thought process to AB’s analysis of the global warming data, it is clear that time is simply a proxy for some underlying causal variable.  We can assume that the future can be extrapolated from the past, if and only if the time-dependence of key causal variables affecting global warming in the past is similar to that expected for the future.  In my view, this is the fundamental flaw in AB’s analysis.  The IPCC models have various scenarios that describe how CO2 emissions and other drivers of global warming can change with time as compared to current conditions.  This is what is missing in the AB analysis.  Not only that, but AB also appear to be unaware of the fact that all physics-based IPCC models are calibrated to past temperature trends and are consistent with the data.  The divergence in their projections is caused by different assumptions about future conditions with respect to CO2 emissions.  A credible model has to explain the past before it can be applied for predicting the future.

In summary, as the simple S&P 500 index example is intended to show, past is prologue only if “before” and “after” operating conditions are similar.  By most accounts, that is not the case with our understanding of how climate is changing.  The “climate-knowledge-free-statistics” based simplistic AB analysis fails to properly take into account the influence and time-dependent behavior of the relevant causal variables, thus leading to results that are incorrect and misleading.  If only AB had applied the same analytical rigor to this problem as for an investment decision!

POSTSCRIPT:  AB’s central premise (i.e., the future can be extrapolated from the past) also brings to mind the simplest weather prediction model – which posits that the weather tomorrow will be the same as that today.  Suffice it to say, in the words of the inimitable H.A.L. Mencken, “for every complex problem there is a solution which is simple, neat and wrong”.  Caveat Emptor. 

1 comment:

  1. It's certainly true that simple extrapolation of the past is not always an accurate forecast of the future. That was not the point of the essay, and certainly not a central premise of it. The point of the essay was that people use the temperature record of recent past, whether the NASA-GISS data series shown or simple consequences of warming or the fact that 2014 was by some measures the warmest year in the dataset, as support for models that predict quite different future behavior. In many cases they even skip the intermediate step and cite the temperature record or the fact of warming as direct support for the idea of catastrophic warming in the near future.

    The paper offers no opinion about whether the future is likely to be like the past, or will conform to model predictions or something else. It only states that the first two options are exclusive.

    To use your S&P500 analogy, suppose there is a market prognosticator who claims the S&P500 will increase next year at three times its long-term historical rate. That's fine, he might be right or wrong. Certainly you are correct that the history does not disprove his claim. But now suppose there are a lot of articles showing the history as proof that the prognosticator must be right. You'd scratch your head and maybe write an essay explaining the error.

    The S&P500 is a poor example for your point for another reason. All those investment managers doing "due diligence about future conditions" consistently make worse forecasts than just extrapolating the long-term trend. You think a "reasonable analysis" requires predicting all sorts of financial variables, but the clear evidence is that those predictions lead to worse predictions, not better.

    You are correct, up to a point anyway, that the models predicting rapid future warming are consistent with the history of mild warming. "Up to a point" because that agreement is built into them and in many cases models have been repeatedly retrofitted to maintain agreement as new data comes out inconsistent with past predictions. Now that doesn't prove that the models are wrong, modeling is hard and 25 years is not a long time to judge them given the complexity of climate. But it also certainly doesn't prove that they're right.

    If the future is like the past, warming is at most a mild problem for the rest of this century. If warming accelerates, or if the Earth changes more in response to warming than it has in the past, warming could be a major problem in the next few decades. Lots of people think the latter is the case, which should imply they think the future will not be like the past, which should imply that they don't use the past as evidence that they are correct.

    ReplyDelete