PROPERTIES OF HIGHER CRITICISM UNDER STRONG DEPENDENCE

By Peter Hall, Jiashun Jin. 2008

70 citations recorded by the source corpus.

The problem of signal detection using sparse, faint information is closely related to a variety of contemporary statistical problems, including the control of false-discovery rate, and classification using very high-dimensional data. Each problem can be solved by conducting a large number of simultaneous hypothesis tests, the properties of which are readily accessed under the assumption of independence. In this paper we address the case of dependent data, in the context of higher criticism methods for signal detection. Short-range dependence has no first-order impact on performance, but the situation changes dramatically under strong dependence. There, although higher criticism can continue to perform well, it can be bettered using methods based on differences of signal values or on the maximum of the data. The relatively inferior performance of higher criticism in such cases can be explained in terms of the fact that, under strong dependence, the higher criticism statistic behaves as though the data were partitioned into very large blocks, with all but a single representative of each block being eliminated from the dataset.

Subfields: Statistics and Probability, Signal Processing

View Paper: https://www.semanticscholar.org/paper/349f97c405795f8eb858067f089dbbf48c523b4d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Signal Processing

Citations in the corpus

Tests alternative to higher criticism for high-dimensional means under sparsity and column-wise dependence (2013, methodology): “Hall and Jin (2008) and Delaigle and Hall (2009) studied the detection boundary for dependent data and Cai and Wu (2012) studied the boundary for detecting mixtures with a general known distribution.”
Asymptotic optimality of the Westfall--Young permutation procedure for multiple testing under dependence (2011, background): “The effect of dependence on the power of Higher Criticism was examined in Hall and Jin [16, 17].”
Optimal detection of heterogeneous and heteroscedastic mixtures (2011, methodology): “Also, a special case of the above model was studied in Hall and Jin (2008, 2010), where σ= 1 so that the model is homoscedastic in a sense.”
Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism (2010, background): “[17, 18] have recently explored the case where the noise may be correlated, i.”
Two‐sample test of high dimensional means under dependence (2014, methodology): “Similar conditions have been imposed in Hall and Jin (2008), Hall and Jin (2010) and Arias-Castro, Candès and Plan (2011).”
Two Sample Tests for High Dimensional Covariance Matrices (2012, methodology): “…distributions either in the context of multiple testing, as in van der Laan and Bryan (2001), Donoho and Jin (2004), Fan, Hall and Yao (2007) and Hall and Jin (2008), or in This is an electronic reprint of the original article published by the Institute of Mathematical Statistics in The Annals…”
Tests for High-Dimensional Covariance Matrices (2010, background): “…in the context of multiple tests as in van der Laan and Bryan (2001), Donoho and Jin (2004), Fan, Hall, and Yao (2007), Kosorok and Ma (2007), and Hall and Jin (2008), or in the context of simultaneous multivariate testing as in Bai and Saranadasa (1996), Schott (2007), and Chen and Qin (2010).”