Confidence Intervals

A point estimate without an interval is a number pretending to be a fact. A confidence interval attaches a calibrated statement of uncertainty to an estimate, but the calibration is about the procedure, not the parameter, and confusing the two is the most common statistical error in finance. Getting the interpretation exactly right, and knowing that intervals and tests are two views of one object, is the core skill here.

The definition, and what it does not say

A $1-\alpha$ confidence interval is a pair of random endpoints $[\hat L, \hat U]$, computed from the data, such that

$\mathbb{P}\big(\hat L \le \theta \le \hat U\big) = 1 - \alpha.$

The probability is over the randomness of the data, over hypothetical repetitions of the whole experiment. The correct reading: "if I repeated this study many times, 95% of the intervals I construct this way would cover the true $\theta$." The incorrect reading, which almost everyone slips into: "there is a 95% probability that $\theta$ lies in this particular interval." Once the data are in hand, $\theta$ is a fixed (unknown) number and the realized interval is fixed; it either contains $\theta$ or it does not, there is no probability left. The 95% is a property of the method's long-run coverage. (The statement people actually want, probability about $\theta$ given this data, is a credible interval, and it requires a prior; see Bayesian Inference.)

Construction: pivot or invert a test

The standard route uses a pivotal quantity, a function of data and parameter whose distribution does not depend on unknowns. For a normal mean with estimated variance, the pivot is

$T = \frac{\bar x - \mu}{\hat\sigma/\sqrt n} \sim t_{n-1}.$

Since $\mathbb{P}(-t_{1-\alpha/2} \le T \le t_{1-\alpha/2}) = 1-\alpha$ regardless of $\mu$, rearranging the inequality to isolate $\mu$ gives

$\bar x \pm t_{1-\alpha/2}\,\frac{\hat\sigma}{\sqrt n}.$

The general large-sample template, courtesy of the asymptotic normality of most estimators, is $\hat\theta \pm z_{1-\alpha/2}\cdot \operatorname{se}(\hat\theta)$. When no clean pivot exists, the Sharpe ratio, a maximum drawdown, a correlation, you resample; see Bootstrap and Resampling.

Duality with hypothesis testing

Confidence intervals and hypothesis tests are the same object seen from two sides. The duality theorem: a value $\theta_0$ lies in the $1-\alpha$ confidence interval if and only if a level-$\alpha$ test of $H_0: \theta = \theta_0$ fails to reject. So a confidence interval is exactly the set of null values you would not reject:

$\text{CI}_{1-\alpha} = \{\theta_0 : \text{test of } H_0:\theta=\theta_0 \text{ not rejected at level } \alpha\}.$

This makes the interval far more informative than a single test. "Is beta different from 1?" is one test; the interval $[0.7, 1.4]$ answers that (yes, 1 is inside, don't reject) and every other null at once, and shows the magnitude of your uncertainty. Reporting an interval instead of a bare "significant / not significant" is almost always the better practice, a point emphasized throughout modern statistics. See Hypothesis Testing.

Worked example: the interval on a Sharpe ratio

A strategy realizes an estimated annual Sharpe $\widehat{\text{SR}} = 0.8$ over $n$ years of monthly data. Under i.i.d. normal returns the standard error of the Sharpe estimate is approximately

$\operatorname{se}(\widehat{\text{SR}}) \approx \sqrt{\frac{1 + \tfrac12 \widehat{\text{SR}}^2}{N}},$

where $N$ is the number of return observations. With 3 years of monthly data, $N = 36$, $\operatorname{se} \approx \sqrt{(1 + 0.32)/36} \approx 0.19$. The 95% interval is $0.8 \pm 1.96(0.19) \approx [0.42,\ 1.18]$, the data are consistent with a mediocre 0.42 and an excellent 1.18. Even over 10 years ($N=120$), the interval is roughly $[0.60, 1.00]$. The lesson is uncomfortable and important: Sharpe ratios estimated from a few years of data carry huge uncertainty, and any allocation decision that treats a point Sharpe as known is ignoring most of what it doesn't know.

Failure modes in financial data

• Autocorrelation understates the width. The $\hat\sigma/\sqrt n$ standard error assumes independence. Serially correlated returns (momentum, illiquidity smoothing, overlapping returns) make the true standard error larger, so naive intervals are too narrow, falsely precise. Use HAC standard errors. See Autocorrelation and Serial Correlation.
• Fat tails and skew break the normal pivot. The $t$-interval assumes approximate normality of the estimate; for heavy-tailed returns the coverage of a nominal 95% interval can be well below 95%, especially for variance, correlation, and Sharpe. Bootstrap intervals adapt better.
• Non-stationarity voids the target. An interval estimates a fixed parameter. If the true beta or vol is drifting, the interval covers a moving target and its coverage guarantee is meaningless. See Stationarity.
• Selection. An interval reported only because the estimate looked good (the best of many backtests) has none of its nominal coverage, the same data-snooping poison as in multiple testing. See p-values and Multiple Testing.

In interviews

The single most common statistics-interview trap is the interpretation question: "what does a 95% confidence interval mean?" The correct answer is about the procedure's long-run coverage over repeated samples, not a 95% probability that the parameter is in the realized interval. Be ready to state the duality with testing (the CI is the set of non-rejected nulls) and to construct a normal-mean interval by pivoting the $t$-statistic. A strong follow-up is the Sharpe-ratio width computation above, showing that a few years of data leave the Sharpe barely distinguishable from zero is exactly the kind of quantitative humility desks want to hear. If asked for "the probability the parameter is in this interval," pivot to the Bayesian credible interval and note it needs a prior.