Fisher information, how much a data point tells you

Question

Interviewers love to ask why some parameters are "easy" to estimate and others are not, given the same sample size.

Define Fisher information, explain how it sets the precision of a maximum likelihood estimate, and interpret it geometrically.

Accepted Answer

Definition. For a single observation with log-density \ell(	heta) = \ln f(x;	heta), the Fisher information is I1(	heta) = \mathbb{E}\!\left[\left(\frac{\partial \ell}{\partial	heta}ight)^2ight] = -\,\mathbb{E}\!\left[\frac{\partial^2 \ell}{\partial	heta^2}ight]. The two forms are equal under mild regularity. The first says information is the variance of the score (the derivative of the log-likelihood); the second says it is the expected curvature of the log-likelihood. Why it sets precision. The MLE from n i.i.d. observations is asymptotically normal: \sqrt{n}\,(\hat	heta - 	heta…

Fisher information, how much a data point tells you

Your answer

More Statistics questions