Control of the mean number of false discoveries, Bonferroni and stability of multiple testing

By Alexander Gordon, Galina V. Glazko, Xing Qiu, Andrei Yakovlev. 2007

188 citations recorded by the source corpus.

The Bonferroni multiple testing procedure is commonly perceived as being overly conservative in large-scale simultaneous testing situations such as those that arise in microarray data analysis. The objective of the present study is to show that this popular belief is due to overly stringent requirements that are typically imposed on the procedure rather than to its conservative nature. To get over its notorious conservatism, we advocate using the Bonferroni selection rule as a procedure that controls the per family error rate (PFER). The present paper reports the first study of stability properties of the Bonferroni and Benjamini--Hochberg procedures. The Bonferroni procedure shows a superior stability in terms of the variance of both the number of true discoveries and the total number of discoveries, a property that is especially important in the presence of correlations between individual $p$-values. Its stability and the ability to provide strong control of the PFER make the Bonferroni procedure an attractive choice in microarray studies.

Subfields: Management Science and Operations Research, Molecular Biology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/44ad82804c2acab9e305de3933a78c59108de02c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

In Molecular Biology

In Statistics and Probability

Citations in the corpus

Bon-EV: an improved multiple testing procedure for controlling false discovery rates (2017, methodology): “…to Benjamini-Hochberg’s procedure in terms of variability of total number of discoveries, and equivalent power to Benjamini-Hochberg’s procedure, when used to control the expected number of false discoveries (EV, where V stands for the number of false discoveries) at a user-specified level [10].”
Multiple Novel Loci are Associated with Indices of Renal Function and Chronic Kidney Disease (2009, background): “5 million tests conducted and corresponds to one or less expected false-positive finding”
Genome-wide association study of blood pressure and hypertension (2009, background): “5 × 10 6 tests (the total number of imputed SNPs), this threshold means that the expected number of false-positive results is ≤1; the validity of this bound is not affected by correlation between test statistic”
A Factor Model Approach to Multiple Testing Under Dependence (2009, methodology): “Recent studies, many of them listed in Gordon et al. (2007), also suggest that high correlations among test statistics affect a strong control of the actual proportion of false discoveries.”