Multicollinearity

Multicollinearity is the problem of regressors that carry overlapping information, value and quality, size and liquidity, ten macro factors that all proxy the business cycle. Unlike heteroskedasticity or autocorrelation, it does not violate any Gauss–Markov assumption (short of perfect collinearity, which breaks rank), so OLS stays BLUE. The damage is subtler and more practical: the estimator becomes so high-variance that individual coefficients are unstable, insignificant, and sign-flippy, even while the model as a whole fits and predicts fine. Understanding why is a clean exercise in the geometry of $X^\top X$.

The mechanism: a near-singular $X^\top X$

Recall $\operatorname{Var}(\hat\beta\mid X) = \sigma^2(X^\top X)^{-1}$. When two or more columns of $X$ are highly correlated, $X^\top X$ is nearly singular, its smallest eigenvalues approach zero, so its inverse has huge entries, and the coefficient variances explode. In the limit of perfect collinearity a column is an exact linear combination of others, $X^\top X$ is singular, the inverse does not exist, and $\hat\beta$ is not even identified (infinitely many $\beta$ fit equally well). Near-collinearity is that catastrophe approached continuously: the estimator is defined but enormously sensitive.

The eigenvalue view is the cleanest. Write the eigendecomposition $X^\top X = \sum_j \lambda_j v_j v_j^\top$; then $\operatorname{Var}(\hat\beta) = \sigma^2\sum_j \lambda_j^{-1} v_j v_j^\top$. Directions in regressor space with tiny variance $\lambda_j$ contribute $1/\lambda_j$, a large number, to the coefficient covariance. Collinearity is precisely the presence of near-zero eigenvalues, i.e. directions along which the regressors barely move. The condition number $\kappa = \sqrt{\lambda_{\max}/\lambda_{\min}}$ summarizes it; $\kappa > 30$ is a common warning threshold.

The variance inflation factor

To quantify collinearity coefficient-by-coefficient, regress each regressor $x_j$ on all the others and take its $R_j^2$. The variance of $\hat\beta_j$ can be written

$\operatorname{Var}(\hat\beta_j) = \frac{\sigma^2}{(n-1)\,\widehat{\operatorname{Var}}(x_j)}\cdot\underbrace{\frac{1}{1 - R_j^2}}_{\text{VIF}_j}.$

The variance inflation factor $\text{VIF}_j = 1/(1 - R_j^2)$ is the multiplicative penalty collinearity imposes on that coefficient's variance. If $x_j$ is 90% explained by the others, $R_j^2 = 0.9$ and $\text{VIF}_j = 10$, the standard error is $\sqrt{10} \approx 3.2$ times what it would be if $x_j$ were orthogonal. Rules of thumb: $\text{VIF} > 5$ is worth noting, $\text{VIF} > 10$ is serious. The VIF makes the abstract "$X^\top X$ near-singular" into an actionable per-factor diagnostic.

The symptoms

• Large standard errors, insignificant $t$-stats on individual coefficients, despite a high overall $R^2$ and a significant $F$-test. Jointly the regressors explain $y$; individually none can claim credit.
• Unstable, sign-flipping coefficients. Dropping one observation, adding a month of data, or swapping a near-duplicate regressor swings the estimates wildly, sometimes reversing signs. The coefficients are not robustly estimable.
• Coefficients with the "wrong" sign or implausible magnitudes, because the estimator is trying to difference two nearly identical columns.

Crucially, none of this hurts prediction of $y$ within the span of $X$, fitted values $\hat y$ are stable even when $\hat\beta$ is not. Multicollinearity is a problem for interpreting individual coefficients, not for forecasting the target.

Worked example: two nearly identical factors

Regress returns on two value signals, book-to-market ($x_1$) and earnings yield ($x_2$), which correlate at $0.95$. Then $R_1^2 \approx 0.90$, so $\text{VIF} \approx 10$ and each coefficient's standard error is inflated ~3×. You might see $\hat\beta_1 = +1.8\ (t=0.9)$ and $\hat\beta_2 = -1.1\ (t=-0.6)$, both insignificant, opposite signs, and the difference between two large noisy numbers. Yet a regression on a single combined value factor gives a clean, significant, stable coefficient. The information is real; splitting it across two collinear proxies just scatters it into noise.

The cures

• Drop or combine redundant regressors, use one value factor, not five proxies. Simplest and often best.
• Principal components / PCA, replace collinear regressors with their leading orthogonal components, killing the small-eigenvalue directions by construction (principal component regression). See PCA (Principal Component Analysis).
• Regularization, ridge adds $\lambda I$ to $X^\top X$, lifting the small eigenvalues off zero and stabilizing the inverse; it is the single most direct antidote and trades a little bias for a large variance reduction. LASSO instead selects one of a collinear group and zeroes the rest. See Ridge and LASSO Regularization.
• More or better-designed data, collinearity is a data problem (regressors that don't vary independently); an experiment or instrument that moves one factor without the other resolves it, though that is rarely available in observational market data.

Failure modes in financial data

• The factor zoo. Style factors are drenched in mutual correlation (value–quality, momentum–low-vol, size–liquidity); running them all in one regression produces uninterpretable, unstable loadings. Orthogonalize or regularize.
• Macro regressors. GDP growth, industrial production, employment, and PMI all track one business cycle; a kitchen-sink macro regression is near-rank-deficient.
• Dummy variable trap. Including a full set of category dummies and an intercept creates perfect collinearity (the dummies sum to the intercept), the inverse simply fails; drop one category.
• Lagged predictors. Highly persistent regressors and their own lags are strongly collinear, a common issue in return-predictability regressions.

In interviews

The key distinction to state crisply: multicollinearity does not bias OLS and does not hurt prediction, it inflates the variance of individual coefficients, making them unstable and insignificant despite a good overall fit. Be able to define the VIF, $\text{VIF}_j = 1/(1-R_j^2)$, interpret it ($\text{VIF}=10$ ⇒ SE inflated ~3×), and connect it to a near-singular $X^\top X$ with small eigenvalues blowing up $(X^\top X)^{-1}$. Name the cures, drop/combine, PCA, ridge, and know the punchline that ridge's $\lambda I$ literally regularizes the small eigenvalues. A classic trap question is the dummy-variable / perfect-collinearity case; recognizing that the intercept plus all dummies is exactly singular is the tell of someone who understands the linear algebra.