The jackknife, estimating bias by leaving one out
Before the bootstrap there was the jackknife: a resampling method that estimates an estimator's bias and variance by systematically leaving out one observation at a time.
Explain how the jackknife estimates bias, give the bias-corrected estimator, and state when it fails.
Your answer
This one is open-ended. Work it through, then check your reasoning against the full solution.