Testing 1000 signals, false discovery rate vs Bonferroni

Question

You test m = 1000 candidate signals at \alpha = 0.05. Bonferroni demands p < 0.05/1000 = 5	imes10^{-5}, which rejects almost nothing. What error rate does Benjamini–Hochberg control instead, and how does the procedure work?

Accepted Answer

Two different error rates. Let V = number of true nulls rejected (false discoveries) and R = total rejections. - FWER (family-wise error rate, what Bonferroni controls) = \mathbb{P}(V \ge 1), the chance of any false positive. Extremely strict when m is large. - FDR (false discovery rate) = \mathbb{E}\!\left[\dfrac{V}{R}\right], the expected proportion of your discoveries that are false. If you make 50 discoveries and control FDR at 0.05, you expect about 2.5 of them to be spurious. This is exactly the right notion for signal screening, where a few false leads are acceptable. Benjamini–Hochber…

Testing 1000 signals, false discovery rate vs Bonferroni

Your answer

More Statistics questions