Separating the Shudder from the Shock: Bipower Variation

Markets move in two fundamentally different ways.

There is the churn: the continuous, restless, high-frequency shuffling of prices as buyers and sellers trade against each other. It is always there, sometimes gentle, sometimes frantic, but it is smooth in the sense that the price gets from A to B by passing through everything in between.

Then there is the jump: a central bank announcement, a shock earnings release, a war. The price does not travel from A to B. It teleports. One instant it is at 100, the next it is at 94, and it was never at 97.

These are different phenomena with different causes, different persistence, and different implications for anyone holding risk. And realized volatility hopelessly mixes them together. Add up squared five-minute returns and you get a single number that contains both the day's ordinary churn and any teleportation that happened to occur.

In 2004, Ole Barndorff-Nielsen and Neil Shephard published the trick that pulls them apart.

The problem: one number, two very different stories

Suppose realized volatility on Tuesday comes in at double its usual level. What happened?

Story one: the market was nervous all day. Every five-minute bar was jumpy. Volatility is genuinely elevated, and it will probably still be elevated tomorrow, because volatility clusters.

Story two: the market was perfectly calm all day except for one moment when a surprise announcement dropped and the price gapped violently. Then it went back to being calm. The elevated number is a one-off. Tomorrow will likely be normal.

These demand completely different responses. If you cannot tell them apart, your volatility forecast will be wrong in both cases: too low after a genuinely turbulent day, and far too high after a calm day with one jump in it. But realized volatility gives you the same number for both.

The key idea via analogy: the trick of multiplying neighbours

Here is the beautiful part, and it is genuinely clever.

Realized volatility squares each return. Squaring is brutal on big numbers: a return ten times as large contributes a hundred times as much. So a single jump dominates the sum.

Barndorff-Nielsen and Shephard proposed something different. Instead of squaring each return, take the absolute size of each return and multiply it by the absolute size of the return immediately before it. Then add up all those products. They called it bipower variation, "bi" because each term involves two returns.

Why does this help? Think about what a jump looks like at five-minute resolution. It is one enormous return sitting in a sea of small ones. Now, in the bipower sum, that enormous return never appears alone. It always appears multiplied by its neighbour, and its neighbour is a perfectly ordinary tiny return. Big times tiny equals moderate. The jump's contribution gets crushed.

Whereas on a genuinely turbulent day, all the returns are large, so every product is large-times-large, and the sum reflects that properly.

The result is a measure that captures the continuous churn while being largely blind to jumps. And they proved this rigorously: as you sample more finely, bipower variation converges to the volatility of the continuous part alone, jumps and all, entirely ignored.

The payoff: subtract to find the jumps

Once you have both quantities, the rest writes itself.

• Realized volatility measures the churn plus the jumps.
• Bipower variation measures the churn alone.
• The difference between them measures the jumps.

You now have a model-free estimate of exactly how much of today's market movement was continuous grinding and how much was discontinuous shock. Feed that difference into a statistical test and you can ask, day by day, "did a jump actually happen today, or is this just noise?" and get a defensible yes or no.

This is a genuinely remarkable thing to be able to do without assuming any model of how prices behave.

Why it mattered

• It gave the field a jump detector. Before this, whether a given day contained a jump was a matter of eyeballing. Afterward it was a hypothesis test. Researchers immediately began cataloguing jumps and asking what causes them, with macroeconomic news announcements emerging as a major answer.
• It improved volatility forecasts. The follow-up work by Andersen, Bollerslev and Diebold showed that if you split realized volatility into its continuous and jump pieces and forecast them separately, you forecast better. The continuous part is highly persistent. The jump part is almost not persistent at all. Lumping them together is a modelling error.
• It matters for risk and for options. Jump risk and diffusive risk are priced differently and hedged differently. A portfolio can be perfectly delta-hedged against churn and still be destroyed by a gap. Being able to measure the two separately is a prerequisite to managing them separately.
• It is robust by design. The bipower construction is a wonderful piece of statistical engineering, achieving robustness to outliers through nothing more elaborate than multiplying neighbouring terms.

The honest limitations

• Microstructure noise breaks it. Bipower variation is, if anything, more sensitive to high-frequency noise than realized volatility, because the bid-ask bounce creates spurious alternation in returns that the neighbour-multiplication picks up. You cannot sample too finely.
• Consecutive jumps defeat the trick. The whole logic depends on a jump's neighbour being small. If two jumps land back to back, or if a jump is followed immediately by turbulent trading, big-times-big survives the multiplication and gets counted as churn.
• Jumps at the boundary of the day are awkward. Overnight gaps, which are arguably the most important jumps in equity markets, sit outside the intraday window entirely.
• The test is a hypothesis test. It has false positives and false negatives, and its performance depends on sampling frequency and on the size of the jump. Small jumps are essentially invisible to it.
• "Jump" versus "burst of volatility" is philosophically slippery. A price that moves very fast but continuously over thirty seconds is, at five-minute sampling, indistinguishable from a jump. Whether the distinction is real or an artefact of your sampling frequency is a fair question.

The one-line takeaway

By multiplying each return by its neighbour instead of by itself, Barndorff-Nielsen and Shephard built a volatility measure that ignores sudden jumps and sees only the market's continuous churn, which means subtracting it from ordinary realized volatility reveals, for the first time and without assuming any model, exactly how much of a day's movement was a shock rather than a shudder.