Why divide by n − p in regression residual variance?

Question

In a linear regression with p fitted coefficients (intercept included), the residual variance is estimated as \hat\sigma^2 = \frac{1}{n-p}\sumi \hat{e}i^2, where \hat ei are the residuals. Why the divisor n - p, and what does it become for simple linear regression? How does this relate to the sample-variance n - 1?

Accepted Answer

The degrees-of-freedom accounting. Fitting p coefficients forces the residuals to satisfy p linear constraints (for example, with an intercept the residuals sum to zero, and each regressor is orthogonal to them). That leaves only n - p genuinely free residuals. It can be shown that \mathbb{E}\left[\sumi \hat ei^2\right] = (n - p)\sigma^2, so dividing by n - p makes \hat\sigma^2 unbiased. Residuals measured around a fitted surface are systematically smaller than the true errors, by exactly p observations' worth of variance. Simple linear regression. With a slope and an intercept, p = 2, so you…

Why divide by n − p in regression residual variance?

Your answer

More Statistics questions