Separating the Signal from the Cycle: the Hodrick-Prescott Filter

Look at a chart of any economic series. Output, employment, industrial production. You will see two things happening at once. There is a long, slow upward march, which is growth. And there are shorter wobbles around that march, which are the business cycle.

Most of macroeconomics is about the wobbles. But to study the wobbles, you first have to remove the march, and there is no law of nature that tells you where one ends and the other begins. The trend is not observable. It is a modelling choice.

Hodrick and Prescott proposed a way of making that choice. Their filter became so ubiquitous in macroeconomics that for two decades "detrending the data" and "applying the HP filter" were essentially synonyms. (The paper circulated as a working paper from 1980 onward and was hugely influential long before its formal 1997 publication.)

The problem: every simple answer is obviously wrong

The naive approach is to fit a straight line, call that the trend, and call the deviations from it the cycle. This fails immediately. Economies do not grow along a straight line for fifty years. Growth rates change: post-war booms, productivity slowdowns, technology accelerations. A straight-line trend will show the economy as being persistently "above trend" for a decade and then persistently "below trend" for the next decade, which is not a business cycle, it is a failure of your trend.

The other extreme is to let the trend be as wiggly as you like. But if the trend can wiggle freely, it will simply trace the data exactly, and there will be no cycle left at all. You have explained everything and learned nothing.

So the trend must be flexible enough to bend, but not so flexible that it absorbs the cycle. How much flexibility? That is the whole question, and it has no objective answer.

The key idea via analogy: bending a flexible ruler through the data

Imagine you are drawing a line through a scatter of points using a thin, springy plastic ruler. You want the ruler to pass close to the points, so you bend it. But the ruler resists bending: it has a natural stiffness that wants it to stay straight.

The line you end up with is a compromise between two competing forces:

1. Get close to the data. Every point the ruler misses is a cost.
2. Do not bend too sharply. Every kink in the ruler is also a cost.

A very floppy ruler will snake through every single point, and the "cycle" (the gaps between ruler and points) will be zero. A very stiff ruler will stay nearly straight and the cycle will contain all the wiggle.

The Hodrick-Prescott filter is exactly this, made mathematical. You choose a stiffness parameter, universally called lambda, that sets the exchange rate between the two costs. Then the filter solves for the trend line that best balances them. The trend is the ruler. The cycle is everything the ruler failed to catch.

That is the entire idea. It is simple, it is fast, it always produces an answer, and it produces an answer that looks exactly like what a macroeconomist expects a trend and a cycle to look like. That combination is why it conquered the field.

The choice of stiffness

Everything hinges on lambda. Set it to zero and the trend is the data itself. Set it to infinity and the trend is a straight line. In between, you get something reasonable.

Hodrick and Prescott suggested a value of 1600 for quarterly data. This number is now recited in macroeconomics classrooms like a physical constant. Where did it come from? Their reasoning was roughly this: a 5% deviation from trend in a quarter is about as "big" a cyclical event as a change in the trend growth rate of one-eighth of a percent per quarter. Ratio those, square it, and you get 1600.

It is a judgement call dressed as a calibration. It has been enormously consequential precisely because so many people accepted it without examining it.

Why it mattered

• It gave macroeconomics a common language. Once everyone detrended their data the same way, results became comparable across papers. The stylised facts of the business cycle (consumption is smoother than output, investment is more volatile, employment is procyclical) are all statements about HP-filtered data.
• It was the enabling tool of real business cycle theory. Prescott's own research programme, which modelled business cycles as the optimal response of an economy to technology shocks, needed a way to extract "the cycle" from data to compare against model simulations. The filter and the theory grew up together.
• It is fast, simple, and always works. Never underestimate this. A tool that requires no judgement, runs instantly, and never fails to produce a plausible-looking answer will be adopted far more widely than a better tool that requires care.
• It shows up in finance too. Detrending a price series, extracting a slow-moving component from volatility, separating a signal into "level" and "deviation from level" for a mean-reversion trade: these are all HP-filter-shaped problems, and it is used for all of them.

The honest limitations

This is where the story gets interesting, because the criticisms of the HP filter are severe, well-established, and largely ignored in practice.

• It invents cycles that are not in the data. This is the most damning result. If you apply the HP filter to a pure random walk, which by construction has no cycle whatsoever, the filter will produce a "cyclical component" with apparent periodicity and structure. The cycles are an artifact of the filter, not a property of the world. Any pattern you find in HP-filtered data may have been manufactured by the filtering.
• The endpoints are unreliable, and the endpoint is where you live. The filter fits the trend using data on both sides of each point. In the middle of the sample, that is fine. At the very end of the sample, there is no future data, so the trend estimate there is much noisier and behaves quite differently. As new data arrives, the filter revises its estimate of the past. This means the "output gap" or "cycle" you computed last quarter can change once you add another observation. For a policymaker, or a trader, this is close to disqualifying: the current value of the thing you care about is the one you can trust least.
• Using it in a backtest is a lookahead disaster. This deserves shouting. If you HP-filter a price series over your whole sample and then trade the deviations, you have used future data to compute today's trend. Your backtest will look spectacular and your live results will not. This is one of the most common and most fatal lookahead biases in quant research, and it is easy to commit without noticing.
• The value of 1600 has no real justification. Different values give materially different cycles, and there is no principled way to choose. Hamilton later showed that a proper statistical framing of the problem implies values wildly different from what everyone uses.
• James Hamilton wrote a paper titled "Why You Should Never Use the Hodrick-Prescott Filter." In 2018. That is a genuine paper title in a top journal, and it is not being coy. His indictment covers the spurious dynamics, the endpoint problem, and the arbitrary lambda, and he proposes a simple regression-based alternative instead. The debate is still live.

The one-line takeaway

Hodrick and Prescott gave economics a fast, simple, universally adopted way to split a series into trend and cycle by bending a springy ruler through the data, and its convenience has kept it in use for decades despite a mountain of evidence that it invents cycles that do not exist and cannot be trusted at exactly the point in time you care about most.