Corrections to LRT on large-dimensional covariance matrix by RMT

By Z. Bai, Dandan Jiang, Jianfeng Yao, Shu-rong Zheng. 2009

266 citations recorded by the source corpus.

In this paper, we give an explanation to the failure of two likelihood ratio procedures for testing about covariance matrices from Gaussian populations when the dimension p is large compared to the sample size n. Next, using recent central limit theorems for linear spectral statistics of sample covariance matrices and of random F -matrices, we propose necessary corrections for these LR tests to cope with high-dimensional effects. The asymptotic distributions of these corrected tests under the null are given. Simulations demonstrate that the corrected LR tests yield a realized size close to nominal level for both moderate p (around 20) and high dimension, while the traditional LR tests with χ 2 approximation fails. Another contribution from the paper is that for testing the equality between two covariance matrices, the proposed correction applies equally for non-Gaussian populations yielding a valid pseudo-likelihood ratio test.

Subfields: Computational Theory and Mathematics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/903747033433056b6fe2bc675432c79dd2c29adf

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Statistics and Probability

Citations in the corpus

On estimation of the noise variance in high dimensional probabilistic principal component analysis (2013, methodology): “known since the work of Amemiya and Anderson (1990). These results are again challenged by high-dimensional data and the classical chi-squared limit is no more valid. Following an approach devised in Bai et al. (2009), we propose a correction to this goodness-of-t test statistic involving our new es”
Estimating structured high-dimensional covariance and precision matrices: Optimal rates and adaptive estimation (2016, methodology): “Various testing methods have been proposed including likelihood ratio test in Bai et al. (2009) and Jiang et al. (2012), largest eigenvalue test in Johnstone (2001), Soshnikov (2002) and Peche (2009), Frobenius distance test in Ledoit and Wolf (2002), Srivastava (2005), Birke and Dette (2005), and…”
Optimal hypothesis testing for high dimensional covariance matrices (2012, methodology): “[2] proposed a corrected LRT when c < 1, and Jiang et al.”
Random matrix theory in statistics: A review (2014, background): “…wopt, is called the optimal allocation, and the curve wToptΣwopt , quantifies the risk associated with the optimal strategy, as a function of mP. Bai et al. (2009) and El Karoui (2009b, 2010c) analyzed this statistical problem from the point of view of RMT when p=n is approximately constant…”
Asymptotic power of sphericity tests for high-dimensional data (2013, methodology): “1 Schott (2006), Bai et al. (2009), Fisher, Sun and Gallagher (2010), Chen, Zhang and Zhong (2010) and Berthet and Rigollet (2012).”
High-dimensional consistent independence testing with maxima of rank correlations (2018, background): “For example, Bai et al. (2009) and Jiang and Yang (2013) derived corrected likelihood ratio tests for Gaussian data.”
Large Sample Covariance Matrices and High-Dimensional Data Analysis (2015, background): “The second direction, following the work of Bai et al. (2009a), is to make systematic corrections to the classical multivariate statistical procedures so that the effect of large dimension is overcome.”
On testing the equality of high dimensional mean vectors with unequal covariance matrices (2014, background): “tests and simulations Due to the fact that the commonly used likelihood ratio test performs badly when dimension is large has been considered in a lot of literature such as Bai and Saranadasa (1996); Bai et al. (2009); Jiang et al. (2012); Jiang and Yang (2013), the discussion of the likelihood rati”
Substitution principle for CLT of linear spectral statistics of high-dimensional sample covariance matrices with applications to hypothesis testing (2014, methodology): “The following result is established in [1].”
Two Sample Tests for High Dimensional Covariance Matrices (2012, methodology): “By employing the theory of large dimensional random matrices, Bai et al. (2009) proposed a correction to the LR statistic and demonstrated that the corrected test is valid under p/n i → c i ∈ (0 , 1).”
Likelihood Ratio Tests for High‐Dimensional Normal Distributions (2013, methodology): “(3) Recently some authors study similar problems under the nonparametric setting, see, e.”
Global Testing and Large-Scale Multiple Testing for High-Dimensional Covariance Structures (2017, background): “Bai et al. (2009) proposed a corrected LRT for which the test statistic has a Gaussian limiting null distribution when p/n → c ∈ (0, 1).”