THE SPECTRUM OF RANDOM INNER-PRODUCT KERNEL MATRICES

By Xiuyuan Cheng, Amit Singer. 2012

96 citations recorded by the source corpus.

We consider n-by-n matrices whose (i,j)-th entry is f(X T i Xj), where X1,...,Xn are i.i.d. standard Gaussian random vectors in Rp, and f is a real-valued function. The eigenvalue distribution of these random kernel matrices is studied at the "large p, large n" regime. It is shown that, when p,n ! 1 and p/n = which is a constant, and f is properly scaled so that V ar(f(XT i Xj)) is O(p 1), the spectral density converges weakly to a limiting density on R. The limiting density is dictated by a cubic equation involving its Stieltjes transform. While for smooth kernel functions the limiting spectral density has been previously shown to be the Marcenko-Pastur distribution, our analysis is applicable to non-smooth kernel functions, resulting in a new family of limiting densities.

Subfields: Artificial Intelligence, Discrete Mathematics and Combinatorics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/170507b1a1784f519372b830d3100fca15bc66b8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Discrete Mathematics and Combinatorics

In Statistics and Probability

Citations in the corpus

The spectral norm of random inner-product kernel matrices (2015, methodology): “In the context of random matrix theory, convergence of the extremal eigenvalues of K(X) was posed as an open question in [20].”
Characterizing and correcting for the effect of sensor noise in the dynamic mode decomposition (2014, background): “We direct the interested reader to relevant results in perturbed SVD’s (Stewart 1991, 2006; Bai and Silverstein 2009), random inner product matrices (Cheng and Singer 2013; Tao and Vu 2012), and POD-type operations on noisy data (Epps and Techet 2010; Singer and Wu 2013; Zhao and Singer 2013).”
Capturing the learning curves of generic features maps for realistic data sets with a teacher-student model (2021, background): “The asymptotic behavior of kernel matrices has indeed been the subject of intense scrutiny [45, 62, 46, 11, 63, 48].”
ournal of Statistical Mechanics : J Theory and Experiment (2019, unknown): “The main focus in this direction has been kernel random matrices and robust statistics models (El Karoui et al., 2010; Cheng & Singer, 2013).”
Generalization of Two-layer Neural Networks: An Asymptotic Viewpoint (2020, methodology): “Techniques used in the current proof are largely borrowed from Cheng and Singer (2013); Hastie et al. (2019), but with one modiﬁcation: our non-linearity function does not have zero-expectation under Gaussian distribution, i.e.”