In-sample R² is 0.6, out-of-sample it's zero, what happened?

Question

You build a return-prediction model with $k$ features on $n$ days of data. In-sample $R^2 = 0.6$ , but on the next month it is essentially $0$ .

What went wrong, and how much in-sample fit would you expect from pure noise?

Accepted Answer

Overfitting. With enough free parameters a model can fit the noise in the training data, not just the signal. In-sample R^2 is a biased, optimistic measure of predictive power because the very same data chose the coefficients that make it look good. How much fit comes from noise alone. Suppose y and all k regressors are independent noise, there is genuinely no relationship. Regressing y on k predictors (plus intercept) with n observations gives, under the null, R^2 \sim 	ext{Beta}\!\left(	frac{k}{2},\ 	frac{n-k-1}{2}ight), \qquad \mathbb{E}[R^2] = \frac{k}{n-1}. So with k = 30 features a…

In-sample R² is 0.6, out-of-sample it's zero, what happened?

Your answer

More Statistics questions