Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses

By Nicolai Meinshausen, J. Rice. 2005

235 citations recorded by the source corpus.

We consider the problem of estimating the number of false null hypotheses among a very large number of independently tested hypotheses, focusing on the situation in which the proportion of false null hypotheses is very small. We propose a family of methods for establishing lower 100(1 - a)% confidence bounds for this proportion, based on the empirical distribution of the p-values of the tests. Methods in this family are then compared in terms of ability to consistently estimate the proportion by letting α → 0 as the number of hypothesis tests increases and the proportion decreases. This work is motivated by a signal detection problem that occurs in astronomy.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/c0111fc2500d09c923ffe2317e2c4adebd912210

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Controlling the familywise error rate with plug‐in estimator for the proportion of true null hypotheses (2009, background): “The focus of Meinshausen and Rice (2006) was on sparsity situations in which the proportion of false null hypotheses is very small; see also Abramovich et al. (2006).”
Only closed testing procedures are admissible for controlling false discovery proportions (2019, methodology): “By Romano et al. (2011) the method is clearly inadmissible as a FWER-controlling method.”
Estimation and Confidence Sets for Sparse Normal Mixtures (2006, methodology): “In this general one-sided Gaussian mixture model, as noted in a similar context by Meinshausen and Rice [14], the upper bound for εn must always be equal to 1: the possibility that εn = 1 can never be ruled out because the nonzero μi can be arbitrarily close to zero.”
STEPUP PROCEDURES CONTROLLING GENERALIZED FWER AND GENERALIZED FDR (2007, background): “SARKAR expected proportion of falsely rejected null hypotheses and which was introduced by Benjamini and Hochberg [1], is the first of these that has received considerable attention [2, 3, 4, 5, 6, 12, 15, 16, 17, 21, 22].”
Some Results on the Control of the False Discovery Rate under Dependence (2007, methodology): “A similar approach is taken in Meinshausen & Rice (2005), who derive conservative estimators for the quantity a under independence.”
A comparison of the Benjamini-Hochberg procedure with some Bayesian rules for multiple testing (2008, background): “Such values of m are of importance in QTL mapping and they have a methodological interest in that the asymptotic results of Genovese and Wasserman [13], Donoho and Jin [6] or Meinshausen and Rice [18] do not yet apply.”
Estimation of a two‐component mixture model with applications to multiple testing (2012, methodology): “The lower bounds that were proposed in both the references approximately satisfy inequality (11) and have the form supt∈.0,1/{Fn.t/ − t − ηn,β δ.t/}=.1 − t/, where ηn,β is a bounding sequence for the bounding function δ.t/ at level β; see Meinshausen and Rice (2006).”
On the performance of FDR control: Constraints and a partial solution (2007, unknown): “[1, 2, 5-10, 12, 15-17] and references therein).”
Asymptotic Properties of Bayes Risk of a General Class of Shrinkage Priors in Multiple Hypothesis Testing Under Sparsity (2013, methodology): “See, for example, Ingster (1997), Donoho and Jin (2004), Meinshausen and Rice (2006), Cai et al. (2007) and Cai and Jin (2010), in this context.”
Higher criticism: $p$-values and criticism (2014, methodology): “…with a completely different class of statistics, the “average likelihood ratio” statistics, which seem to have excellent power, but do not appear to be useful when estimation of the number of non-null distributions (Meinshausen and Rice, 2006) or identification of them is also desirable).”
α‐investing: a procedure for sequential control of expected false discoveries (2008, background): “work, Meinshausen and B¨uhlmann (2004) and Meinshausen and Rice (2006) estimate”
Asymptotic optimality of the Westfall--Young permutation procedure for multiple testing under dependence (2011, background): “Such testing problems arise in many applications, in particular, in the fields of genomics and genome-wide association studies [9, 18, 24], but also in astronomy and other fields [21, 26].”
Estimating the Null and the Proportion of Nonnull Effects in Large-Scale Multiple Comparisons (2006, methodology): “See, for example, [1, 4, 6, 7, 8, 10, 17 , 19].”
Proportion of non‐zero normal means: universal oracle equivalences and uniformly consistent estimators (2008, methodology): “In a sense, our findings in this paper complement with those in Genovese and Wasserman (2004) and Meinshausen and Rice (2006).”
On false discovery control under dependence (2008, background): “As demonstrated in Storey, Taylor and Siegmund [19], Genovese and Wasserman [9], Chi [5] and Meinshausen and Rice [13] among others, the theory of empirical processes plays a useful role in the study of false discovery control.”