The Volatility Models Were Fine, the Scorecard Was Broken

By the late 1990s, GARCH had a public relations disaster on its hands.

The academic literature was full of it, Nobel prizes were on the horizon for it, and practitioners were building risk systems on it. And yet, when researchers ran the obvious test, comparing GARCH's volatility forecast for tomorrow against what actually happened tomorrow, the results were humiliating. The model typically "explained" only a few percent of the variation in the outcome. Five percent. Sometimes less.

Sceptics drew the natural conclusion: these models are elaborate nonsense. All that mathematics, all those Greek letters, and you can barely beat a coin flip.

Torben Andersen and Tim Bollerslev wrote a paper with a title that is essentially a rebuttal in itself, and showed that the sceptics were measuring the wrong thing.

The problem: you cannot grade a forecast against a terrible answer key

Here is the subtle mistake everyone was making. A volatility forecast predicts something you can never actually observe. Volatility is not a number the market prints. It is a hidden quantity, the scale of the random fluctuation, and you only ever see one realisation drawn from it.

So how did people grade the forecast? They used tomorrow's squared return as the "actual" volatility. It is the obvious choice. It is what GARCH is built from. And it is, in a technical sense, unbiased: on average, the squared return equals the true variance.

But unbiased is not the same as accurate. A single squared daily return is an appallingly noisy measure of that day's true volatility. Imagine a day when the market is genuinely churning, high true volatility, but by pure chance the price ends up almost exactly where it started. The squared return is near zero. Your answer key now says "today was dead calm," which is a lie.

Grade a perfectly good forecast against an answer key that is mostly noise, and the forecast will look terrible. Andersen and Bollerslev worked out the arithmetic and showed that even a perfect, oracle-quality volatility forecast would score only around 5% or so when graded this way. The low scores were not evidence against GARCH. They were an inescapable consequence of the ruler being made of rubber.

The key idea via analogy: measuring a child's height with one blurry photo

Suppose you want to know how tall a child is. You have a model that predicts her height at age ten, quite accurately. To check it, you take one blurry photograph from an odd angle and estimate her height from that. The photo estimate is unbiased, on average it is right, but any single photo is wildly off.

Now compare your model's prediction to the photo. They disagree. Do you conclude the model is bad? Of course not. The photo is bad.

The fix is obvious once you say it out loud: take more photographs. Andersen and Bollerslev's fix is exactly that. Instead of measuring the day's volatility with one number, the daily squared return, use high-frequency intraday data. Chop the trading day into short intervals, five minutes say, compute the return over each interval, square them all, and add them up.

That sum, which the world now calls realized volatility, is a vastly better answer key. It uses hundreds of observations per day instead of one. It is not perfect, but it is enormously less noisy, and it is a genuine measurement of what the day's volatility actually was rather than a single blurry draw from it.

What happened when they regraded

They took exchange rate data, built realized volatility from intraday returns, and regraded the standard GARCH forecasts against this better answer key.

The models went from looking useless to looking genuinely good. Once you stop grading against noise, the volatility forecasts turn out to track the true, latent volatility with real accuracy. GARCH had been doing its job all along. Nobody had been able to see it, because the measuring instrument was broken.

Why it mattered

• It rescued a whole literature. This paper stopped a serious and growing scepticism about whether volatility modelling had any practical value at all. The title is not shy about it.
• It launched realized volatility. The idea of summing squared intraday returns to measure a day's volatility is the seed of an entire modern field. If you can measure volatility rather than infer it, you can model it directly with ordinary time series tools. The papers of Andersen, Bollerslev, Diebold and Labys, of Barndorff-Nielsen and Shephard, and of Corsi all follow from this door being opened.
• It taught a general lesson about evaluation. Whenever you are forecasting something you cannot observe, the quality of your proxy determines the quality of your conclusions. This lesson has been relearned painfully in many other corners of quantitative finance. Andrew Patton later made it fully rigorous.
• It changed what practitioners measure. Risk desks and volatility traders now routinely compute realized volatility from intraday data rather than relying on daily closes.

The honest limitations

• The models were still simple. Vindicating GARCH's accuracy is not the same as saying GARCH is the best available model. Once realized volatility existed as a measurement, models built directly on it, such as HAR, went on to beat GARCH comfortably.
• Realized volatility has its own problems. Summing squared five-minute returns sounds clean, but intraday prices are contaminated by market microstructure noise, bid-ask bounce and stale quotes. Sample too finely and you measure the noise instead of the volatility. This problem consumed a decade of subsequent research.
• You need the data. High-frequency data is expensive and, in 1998, was available only for a handful of liquid markets. Many assets still cannot be measured this way.
• Overnight gaps are missed. Realized volatility built from trading-hours returns ignores what happens while the market is closed, which for equities is a substantial chunk of total volatility.
• The result was about accuracy, not economic value. Showing a forecast tracks true volatility is not the same as showing it makes money or improves risk management. That takes further work.

The one-line takeaway

Andersen and Bollerslev showed that standard volatility models were never the problem, the answer key was, and by replacing a single noisy squared daily return with a sum of intraday returns, they simultaneously vindicated GARCH and invented the measurement that would go on to replace it.