Stop Guessing Volatility, Start Measuring It

Every volatility model before this one had to fight the same handicap: volatility is invisible. You never observe it. You observe returns, and you infer volatility from them using a model, GARCH or stochastic volatility or whatever else. All the mathematical complexity of that literature exists to solve one problem: how do you estimate something you cannot see?

In 2003, Torben Andersen, Tim Bollerslev, Francis Diebold and Paul Labys published a paper whose central move is almost impudent. What if you just measured it?

The problem: the whole field was solving the wrong problem

Consider what GARCH actually is. It is an elaborate filtering device. It takes a long history of returns and produces, at each date, a best guess about a hidden state variable. To make that work, you have to assume a specific functional form, estimate parameters by maximum likelihood, and hope your assumptions are right. If they are wrong, your inferred volatility is wrong, and you may never find out.

That entire machine exists because a single day gives you a single return, and one number cannot tell you the scale of the randomness that produced it.

But a single day does not give you one number. It gives you thousands. Trades happen every second. The only reason volatility ever looked invisible is that people were throwing away almost all of the data by using only the closing price.

The key idea via analogy: measuring how far a dog walked

You take your dog for a walk. At the end, the dog is fifty metres from where it started. How far did the dog actually travel?

If you only look at start and end, you would say fifty metres. But you were there. The dog ran in circles, chased a squirrel, doubled back, and covered several kilometres. The net displacement and the total distance travelled are completely different quantities.

A day's stock return is net displacement. Volatility is total distance travelled. If you only look at the close, you cannot tell the difference between a dog that ambled fifty metres in a straight line and a dog that sprinted three kilometres and happened to end up fifty metres away.

Now suppose you tracked the dog with a GPS that pings every five minutes. Add up the distance covered in each interval. That sum is a direct measurement of the total distance, no model required. That is realized volatility: chop the trading day into short intervals, take the return over each, square them, and add them all up.

The authors gave this a proper theoretical footing. Under the standard continuous-time picture of how prices move, that sum converges, as you sample more finely, to the day's true integrated variance. It is not a clever estimator with a standard error. In the limit, it is the quantity. Volatility stops being a hidden state and becomes an observable.

The consequence: volatility becomes an ordinary time series

Once you can see volatility, everything gets easier. You no longer need special-purpose machinery designed to peer through a fog. You have a series of daily numbers, and you can model it with the same boring, well-understood tools you would use for any time series.

That is exactly what they did. Working with exchange rate data, they took the logarithm of daily realized volatility, noticed that it looks strikingly close to a normal bell curve (unlike returns, which are famously fat-tailed), noticed that it has a very long memory, and fitted a simple long-memory time series model to it.

The result: their simple model forecast volatility better than the sophisticated GARCH and stochastic volatility models it was competing against. Not marginally. Comfortably. Decades of increasingly elaborate volatility machinery was outperformed by a straightforward time series model applied to a better-measured quantity.

They also showed you can do the same thing for covariances between assets, not just individual volatilities, which opens the door to measuring the entire correlation structure of a portfolio directly.

Why it mattered

• It reframed the whole field. Volatility modelling stopped being about clever inference and started being about clever measurement. That is a different discipline with different tools.
• It made the forecasting problem tractable. Forecasting an observable series is a solved problem. Forecasting a latent state through a possibly misspecified filter is not.
• It documented the key stylised facts. Realized volatility is approximately log-normal, has extremely long memory, and its logarithm is well behaved. Those three facts drive essentially every realized-volatility model built since, including Corsi's HAR.
• It went straight into practice. Realized volatility is now a standard input on volatility trading desks, in risk systems and in academic datasets. Exchanges and data vendors publish it.

The honest limitations

• The theory assumes prices you cannot observe. The convergence result holds for a frictionless, continuously observed price. Real prices come with a bid-ask spread, discrete ticks, and stale or bunched quotes. Sample too finely, say every second, and your "realized volatility" ends up measuring the bounce between bid and ask rather than the movement of the underlying value. The whole literature on microstructure noise, including the two-time-scales estimator, grew out of this problem.
• The five-minute compromise is a fudge. Practitioners typically sample every five minutes precisely because it is fine enough to be informative and coarse enough to dodge the worst of the noise. It works well, but there is nothing sacred about five minutes, and the right choice varies by asset.
• Jumps are lumped in. Realized volatility measures everything the price did, including sudden discontinuous jumps on news. Whether you want jumps inside your volatility measure depends on what you are doing, and separating them out required later work on bipower variation.
• Closed markets are invisible. For assets that stop trading overnight, realized volatility built from trading hours misses the overnight move entirely. Currency markets, which the paper studies, trade almost continuously and dodge this problem. Equities do not.
• It needs liquidity. In a thinly traded name, high-frequency returns are mostly zeros punctuated by occasional large moves, and the whole approach degrades.

The one-line takeaway

Andersen, Bollerslev, Diebold and Labys pointed out that if you use intraday data, volatility is not hidden at all, it is something you can simply add up and observe, and once you can observe it, a simple time series model beats the elaborate machinery that was invented to guess at it.