Combining two noisy estimates optimally

Question

You have two independent unbiased estimates of the same quantity $\mu$ : one with variance $4$ (noisy), the other with variance $1$ (precise). You combine them as $\hat{\mu} = w X_1 + (1 - w) X_2$ .

What weight $w$ minimizes the variance of the combined estimate, and what is that minimum variance?

Accepted Answer

Setup. Any weighted average w X1 + (1-w)X2 is unbiased for \mu since both pieces are. Its variance, using independence, is \operatorname{Var}(\hat{\mu}) = w^2 v1 + (1 - w)^2 v2, \qquad v1 = 4,\; v2 = 1. Optimal weight. Minimizing over w (take the derivative and set it to zero) gives the inverse-variance rule: weight each estimate in proportion to 1/v, w = \frac{1/v1}{1/v1 + 1/v2} = \frac{v2}{v1 + v2} = \frac{1}{4 + 1} = 0.2. So put weight 0.2 on the noisy estimate and 0.8 on the precise one, the cleaner estimate earns more of the blend. Minimum variance. Substituting back (or using the shortc…

Combining two noisy estimates optimally

Your answer

More Statistics questions