Bias, variance, and irreducible noise in prediction error
Suppose where has mean and variance , independent of everything else. You fit a model on random training data and predict at a fixed point .
Show that the expected squared prediction error at splits into irreducible noise plus bias squared plus variance, and use it to explain overfitting.
Your answer
This one is open-ended. Work it through, then check your reasoning against the full solution.