Deriving the least-squares line by hand

Question

You want to fit a straight line y = a + bx to data (xi, yi) by minimizing the sum of squared vertical residuals. Derive the formulas for the intercept a and slope b from scratch, and note what they reveal.

Accepted Answer

The objective. Minimize Q(a,b) = \sumi (yi - a - b xi)^2 over a and b. Intercept. Set the partial derivative with respect to a to zero: \frac{\partial Q}{\partial a} = -2\sumi (yi - a - b xi) = 0 \implies \bar y = a + b\bar x \implies a = \bar y - b\bar x. This says the fitted line passes through the point of averages (\bar x, \bar y), always. Slope. Set the partial derivative with respect to b to zero and substitute a = \bar y - b\bar x: \frac{\partial Q}{\partial b} = -2\sumi xi (yi - a - b xi) = 0. After plugging in the intercept and simplifying, the cross-terms center the variables and yo…

Deriving the least-squares line by hand

Your answer

More Statistics questions