Why Benjamini-Hochberg is a step-up procedure

Question

A subtlety trips people up on Benjamini-Hochberg (BH): once you find the cutoff rank, you reject everything below it, even a hypothesis whose own p-value sits above its personal threshold.

Work through a BH example with $m = 8$ tests at $\alpha = 0.05$ that shows this, and explain why it is right.

Accepted Answer

Sort the p-values and compute each threshold 	frac{k}{m}\alpha = k \cdot 0.00625: k p{(k)} 	frac{k}{m}\alpha p{(k)} \le 	frac{k}{m}\alpha? ------------ 1 0.001 0.00625 Yes 2 0.010 0.01250 Yes 3 0.013 0.01875 Yes 4 0.030 0.02500 No 5 0.031 0.03125 Yes 6 0.036 0.03750 Yes 7 0.200 0.04375 No 8 0.500 0.05000 No BH rejects hypotheses (1), \dots, (k^\ast) where k^\ast is the largest k that passes. Scanning from the bottom, the largest passing rank is k^\ast = 6 (both k=5 and k=6 pass). So BH rejects the top six signals. The step-up subtlety. Rank 4 (p = 0.030) failed its own threshold (0.030 0.0…

Why Benjamini-Hochberg is a step-up procedure

Your answer

More Statistics questions