Chaos in random neural networks.

By Haim Sompolinsky, A. Crisanti, H. Sommers. 1988

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaos (2019, background): “While we have yet to computationally conﬁrm our deep hypothesis, several works have investigated chaos in neural networks (Kolen & Pollack, 1991; Sompolinsky et al., 1988), although we believe we are the ﬁrst to suggest that chaos may cause gradients to degenerate when computed using…”
Exponential expressivity in deep neural networks through transient chaos (2016, methodology): “Here we attack both central problems of deep neural expressivity by combining Riemannian geometry [16] and dynamical mean field theory [17].”
Mean Field Analysis of Neural Networks: A Law of Large Numbers (2018, background): “ustive list). In fact, mean eld analysis has been actively used for many years to study biological neural networks and physical systems of interacting particles, see for example [8], [15], [21], [4], [24], and the references therein. Upon completion of this work, we became aware of the very recent w”
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks (2018, background): “Indeed several pieces of seminal work used statistical physics (Derrida & Pomeau; Sompolinsky et al., 1988) and Gaussian Processes (Neal, 2012) to show that neural networks exhibit remarkable regularity as the width of the network gets large.”
A Constructive Mean-Field Analysis of Multi-Population Neural Networks with Random Synaptic Weights and Stochastic Inputs (2008, positive): “…Is t s d∫ ⋅F( ) Ws (31) We now introduce for future reference a simpler model which is quite frequently used in the description on neural networks and has been formally analyzed by Sompolinsky and colleagues (Crisanti et al., 1990; Sompolinsky et al., 1998) in the case of one population (P = 1).”
Selection, Tinkering, and Emergence in Complex Networks - Crossing the Land of Tinkering (2002, methodology): “This type of random model has been used in diﬁerent contexts, including ecological (May, 1976; Cohen 1978; Sol¶e et al. , 2002), genetic and metabolic (Kauﬁman, 1962; Kauﬁman, 1993) and neural (Hertz et al. , 1991; Sompolinsky et al. , 1988) networks.”
Linking Connectivity, Dynamics, and Computations in Low-Rank Recurrent Neural Networks. (2017, unknown): “Dynamical equations for these quantities can be derived by extending the classical dynamical mean-field theory (Sompolinsky et al., 1988).”
Recurrent Network Models of Sequence Generation and Memory. (2016, methodology): “Sequences may effectively mediate the short-term memory engaged in these tasks.”
Generating Coherent Patterns of Activity from Chaotic Neural Networks (2009, unknown): “Networks with g < 1 are inactive prior to training, whereas networks with g > 1 exhibit chaotic spontaneous activity (Sompolinsky et al., 1988) that gets more irregular and fluctuates more rapidly as g is increased beyond 1 (we typically use g = 1.5).”
Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons (2014, unknown): “(15) np g © 2 01 4 N at ur e A m er ic a, In c.”
Numerical implementation of dynamical mean field theory for disordered systems: application to the Lotka–Volterra model of ecosystems (2019, background): “[6] H.”