Asymptotic Minimaxity Of False Discovery Rate Thresholding For Sparse Exponential Data

By David L. Donoho, Jiashun Jin. 2006

61 citations recorded by the source corpus.

Control of the False Discovery Rate (FDR) is an important development in multiple hypothesis testing, allowing the user to limit the fraction of rejected null hypotheses which correspond to false rejections (i.e. false discoveries). The FDR principle also can be used in multiparameter estimation problems to set thresholds for separating signal from noise when the signal is sparse. Success has been proven when the noise is Gaussian; see [3]. In this paper, we consider the application of FDR thresholding to a non-Gaussian setting, in hopes of learning whether the good asymptotic properties of FDR thresholding as an estimation tool hold more broadly than just at the standard Gaussian model. We consider a vector Xi, i = 1,...,n, whose coordinates are independent exponential with individual means µi. The vector µ is thought to be sparse, with most coordinates 1 and a small fraction significantly larger than 1. This models a situation where most coordinates are simply ‘noise’, but a small fraction of the coordinates contain ‘signal’. We develop an estimation theory working with log(µi) as the estimand, and use the percoordinate mean-squared error in recovering log(µi) to measure risk. We consider minimax estimation over parameter spaces defined by constraints on the per-coordinate ‘ p norm of log(µi): 1 n ( P n i=1 log p (µ i)) p . Members of such spaces are vectors (µ i) which are sparsely heterogeneous. We find that, for large n and small , FDR thresholding can be nearly minimax, increasingly so as decreases. The FDR control parameter 0 1 prevents near minimaxity. These conclusions mirror those found in the Gaussian case in [3]. The techniques developed here seem applicable to a wide range of other distributional assumptions, other loss measures, and non-i.i.d. dependency structures.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/6db29ebf644209409e33a1d8abc2b24faed987b0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

1. Tommaso Cai
2. Jianqing Fan
3. Jinchi Lv
4. Jiashun Jin
5. Peter Hall
6. Cun-Hui Zhang
7. A. Chakrabarti
8. E.J. Candes

Citations in the corpus

• Optimal rates of convergence for estimating the null density and proportion of nonnull effects in large-scale multiple testing (2010, methodology): “See also Abramovich et al. (2006) and Donoho and Jin (2006) for estimation using a multiple testing approach.”
• Wavelets, Ridgelets, and Curvelets for Poisson Noise Removal (2008, background): “Minimaxity of FDR has also been studied in various settings (see [38], [39] for details).”
• The Bayes oracle and asymptotic optimality of multiple testing procedures under sparsity (2010, background): “We cite a few among many important papers, ([9], [10], [24], [20], [5]).”
• On the performance of FDR control: Constraints and a partial solution (2007, unknown): “[1, 2, 5-10, 12, 15-17] and references therein).”
• Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit (2006, result): “The problem of separating ‘signal’ from ‘noise’ in such problems has generated a large literature including the papers [24, 27, 26, 1, 23 , 37], which influenced our way of thinking.”
• Discovering the false discovery rate (2010, methodology): “(a) increasing its generality (Donoho and Jin, 2006); (b) showing that the FDR can be an effective model selection criterion, as it can be translated into a penalty function (Benjamini and Gavrilov, 2009); (c) investigating the theoretical boundaries of detection of signals in the space that is span”
• A Selective Overview of Variable Selection in High Dimensional Feature Space (2009, background): “See also Efron et al. (2001), Abramovich et al. (2006), Donoho and Jin (2006), and Clarke and Hall (2009).”
• PROPERTIES OF HIGHER CRITICISM UNDER STRONG DEPENDENCE (2008, background): “Work on these respective problems includes that of (1) Benjamini and Hochberg [2], Genovese and Wasserman [11], Donoho and Jin [8], Efron et al. [10] and Storey, Dai and Leek [26]; (2) Swanepoel [27], Efron et al. [10], Cai, Jin and Low [4], Genovese and Wasserman [11], Storey, Dai and Leek [26],…”