The singular values and vectors of low rank perturbations of large rectangular random matrices

By Florent Benaych-Georges, Raj Rao Nadakuditi. 2011

388 citations recorded by the source corpus.

In this paper, we consider the singular values and singular vectors of finite, low rank perturbations of large rectangular random matrices. Specifically, we prove almost sure convergence of the extreme singular values and appropriate projections of the corresponding singular vectors of the perturbed matrix. As in the prequel, where we considered the eigenvalues of Hermitian matrices, the non-random limiting value is shown to depend explicitly on the limiting singular value distribution of the unperturbed matrix via an integral transform that linearizes rectangular additive convolution in free probability theory. The asymptotic position of the extreme singular values of the perturbed matrix differs from that of the original matrix if and only if the singular values of the perturbing matrix are above a certain critical threshold which depends on this same aforementioned integral transform. We examine the consequence of this singular value phase transition on the associated left and right singular eigenvectors and discuss the fluctuations of the singular values around these non-random limits.

Subfields: Discrete Mathematics and Combinatorics, Mathematical Physics, Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Discrete Mathematics and Combinatorics

In Mathematical Physics

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

On the principal components of sample covariance matrices (2014, result): “Moreover, in [9,32] result analogous to those of [8] were obtained for the model from Example (2).”
Generalization in multitask deep neural classifiers: a statistical physics approach (2019, background): “In the low SNR regime, the noisy teacher’s singular values lie in the bulk of the MP sea [25].”
Outlier Eigenvalues for Deformed I.I.D. Random Matrices (2014, methodology): “As in previous works such as [9, 10], we will use the identity, if P,Q> ∈MN,r(C), det(IN + PQ) = det(Ir +QP ).”
Complex diffusion-weighted image estimation via matrix recovery under general noise models (2018, methodology): “at those limits can be used to build optimal shrinkers, to be understood as those that minimize the asymptotic loss among all possible shrinkers, for unitarily invariant costs such as (13). Following Benaych-Georges and Nadakuditi (2012), we can define the D-transform as D f(x) = ( ∫ ∞ −∞ x x2 −η2 W”
A Random Matrix Approach to Neural Networks (2017, methodology): “…ELM_Romain_MSE.tex date: February 20, 2017 Transposed to our present setting, the aforementioned Taylor expansion allows for a consistent approximation Φ̃ of Φ by an information-plusnoise (spiked) random matrix model (see e.g., (Loubaton and Vallet, 2010; Benaych-Georges and Nadakuditi, 2012)).”
Bi-cross-validation for factor analysis (2015, neutral): “See [49, 9, 51] for statements of this result.”
Cleaning large correlation matrices: tools from random matrix theory (2016, background): “Note that the case of outlier singular values and vectors of rectangular randommatrices subject to a low rank perturbation has been considered [170].”
Singular vector and singular subspace distribution for the matrix denoising model (2018, background): “The study on the behavior of the extreme eigenvectors has been mainly focused on the level of the first order limit [10, 11, 20, 25, 33, 55].”
Rate-Optimal Perturbation Bounds for Singular Subspaces with Applications to High-Dimensional Statistics (2016, background): “(2009), Benaych-Georges and Nadakuditi (2012), Shabalin and Nobel (2013) focused on the asymptotic distributions of single singular value and vector when p1, p2 and the singular values grows proportionally. Vu (2011) discussed the squared matrix perturbed by i.i.d Bernoulli matrix and derived an upp”
Statistical inference for principal components of spiked covariance matrices (2020, methodology): “We refer to [14, 13, 21, 26, 69, 16, 27] for more details of such a phenomenon.”
On simplicity and complexity in the brave new world of large-scale neuroscience. (2015, background): “By combining and extending time series random matrix theory [46], low-rank perturbation theory [47], and noncommutative probability theory, [48], we have derived a theoretically predicted phase boundary (black curve in (C) and(D)), that matches simulations.”
Spiked separable covariance matrices and principal components (2019, methodology): “It is commonly used in the statistical literature, for instance [6, 15, 16, 38].”
The Optimal Hard Threshold for Singular Values is $4/\sqrt {3}$ (2014, methodology): “Both follow from recent key results due to [25].”
Finding and Leveraging Structure in Learning Problems (2012, background): “In fact, it is now well understood (see for example the paper of Benaych-Georges and Rao Nadakuditi [26]) that SVD is strongly inconsistent when the signal strength is weak, i.”