Cleaning large correlation matrices: tools from random matrix theory

By Joel Bun, Jean-Philippe Bouchaud, Marc Potters. 2016

302 citations recorded by the source corpus.

This review covers recent results concerning the estimation of large covariance matrices using tools from Random Matrix Theory (RMT). We introduce several RMT methods and analytical techniques, such as the Replica formalism and Free Probability, with an emphasis on the Marchenko-Pastur equation that provides information on the resolvent of multiplicatively corrupted noisy matrices. Special care is devoted to the statistics of the eigenvectors of the empirical correlation matrix, which turn out to be crucial for many applications. We show in particular how these results can be used to build consistent "Rotationally Invariant" estimators (RIE) for large correlation matrices when there is no prior on the structure of the underlying process. The last part of this review is dedicated to some real-world applications within financial markets as a case in point. We establish empirically the efficacy of the RIE framework, which is found to be superior in this case to all previously proposed methods. The case of additively (rather than multiplicatively) corrupted noisy matrices is also dealt with in a special Appendix. Several open problems and interesting technical developments are discussed throughout the paper.

Subfields: Condensed Matter Physics, Economics and Econometrics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Condensed Matter Physics

In Economics and Econometrics

In Statistics and Probability

Citations in the corpus

Autoregressive Convolutional Neural Networks for Asynchronous Time Series (2017, methodology): “applications of Random Matrix Theory in econophysics (Laloux et al., 2000; Bun et al., 2017)) and heavy-tailed distributions (Cont, 2001).”
Ligand biological activity predicted by cleaning positive and negative chemical correlations (2019, background): “This problem is all of the more challenging, as it is often the conﬂuence of different motifs rather than a single motif that drives binding, but correlations are known to be especially sensitive to noise due to undersampling (20–22).”
Statistical inference for principal components of spiked covariance matrices (2020, unknown): “[20] J.”
Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning (2018, methodology): “7 Several overviews of RMT are available [143, 41, 71, 142, 22, 42, 110, 24].”