R² jumped to 0.6, but you added 30 regressors

Question

You fit a regression on n = 100 observations. Starting from a bare model, you add regressors until you have p = 30 of them, and the in-sample R^2 climbs to 0.60. Is that a good model? Compute the adjusted R^2 and explain what it reveals.

Accepted Answer

Why raw R^2 is not evidence. Ordinary least squares can always tune a new coefficient to soak up some chance correlation, so in-sample R^2 only ever rises as you add predictors. With 30 regressors and only 100 data points, a good chunk of that 0.60 is the model fitting noise, not signal. Adjusted R^2 charges for degrees of freedom used: \bar{R}^2 = 1 - (1 - R^2)\,\frac{n - 1}{n - p - 1} = 1 - (1 - 0.60)\,\frac{99}{69} = 1 - 0.40 	imes 1.435 \approx 0.426. So the honest, penalty-adjusted figure is about 0.43, not 0.60. The gap of roughly 0.17 is the "rent" charged for spending 30 degrees of f…

R² jumped to 0.6, but you added 30 regressors

Your answer

More Statistics questions