Eigenvalue spectra of random matrices for neural networks.

By K. Rajan, L. F. Abbott. Physical Review Letters, 2006

320 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Signal Processing

View Paper: https://www.semanticscholar.org/paper/d800e8764a784ca37df42d938260f0d90a2c9347

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Signal Processing

Citations in the corpus

Interactions between Intrinsic and Stimulus-Evoked Activity in Recurrent Neural Networks (2009, background): “…Rajan Department of Neuroscience Department of Physiology and Cellular Biophysics Columbia University College of Physicians and Surgeons New York, NY 10032‐2695 USA and Haim Sompolinsky Racah Institute of Physics Interdisciplinary Center for Neural Computation Hebrew University Jerusalem, Israel”
Training Excitatory-Inhibitory Recurrent Neural Networks for Cognitive Tasks: A Simple and Flexible Framework (2016, background): “The means μ E (excitatory) and μ I (inhibitory) of the gamma distributions are chosen to balance the excitatory and inhibitory inputs to each unit [55], i.e., ∑ j 2 exc | μ j | = ∑ j 2 inh | μ j |, with the overall mean set by the imposed spectral radius ρ .”
Linking Connectivity, Dynamics, and Computations in Low-Rank Recurrent Neural Networks. (2017, unknown): “As previously noticed in (Harish and Hansel, 2015), the radius of the circular compact set r can be computed as in (Rajan and Abbott, 2006; Aljadeff et al., 2015b) by summing the variances of the distributions in every column of Sij .”
Statistical mechanics of complex neural systems and high dimensional data (2013, background): “connectivity obeys Dale’s law, which stipulates that all the outgoing synaptic weights of any neuron have the same sign [95].”
Synchronization in complex networks (2008, background): “The analysis of the spectra of directed and weighed networks[271, 272], which in general are complex, and the study of the impact of the directed characterizations of the networks on synchronization process[96, 158, 44], will be key to understanding dynamical organization in more realistic complex s”
Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning (2018, background): “different Heavy-Tailed Universality classes [46, 32, 18, 25, 20, 5, 111, 7, 38, 17, 90, 8, 99]; other work on RMT approaches [133, 120, 114, 112, 87, 87, 137, 85, 126]; other work on statistical physics approaches [132, 52, 117, 125, 137, 119, 113]; work on fitting to noisy versus reliable signal [1”
Outlier Eigenvalues for Deformed I.I.D. Random Matrices (2014, methodology): “Finally, as in Rajan and Abbott [44] and Tao [51], interesting outliers may appear when ‖A′′ N‖ is of order √ N .”
Long short-term memory and Learning-to-learn in networks of spiking neurons (2018, background): “[6] Jane X Wang, Zeb Kurth-Nelson, Dhruva Tirumala, Hubert Soyer, Joel Z Leibo, Remi Munos, Charles Blundell, Dharshan Kumaran, and Matt Botvinick.”
Lower bounds for the smallest singular value of structured random matrices (2016, methodology): “The limiting spectral distributions and spectral radius for certain structured random matrices have been used to model the dynamical properties of neural networks [1, 25].”