Transition to chaos in random neuronal networks

By Jonathan Kadmon, Haim Sompolinsky. 2015

218 citations recorded by the source corpus.

Firing patterns in the central nervous system often exhibit strong temporal irregularity and heterogeneity in their time averaged response properties. Previous studies suggested that these properties are outcome of an intrinsic chaotic dynamics. Indeed, simplified rate-based large neuronal networks with random synaptic connections are known to exhibit sharp transition from fixed point to chaotic dynamics when the synaptic gain is increased. However, the existence of a similar transition in neuronal circuit models with more realistic architectures and firing dynamics has not been established. In this work we investigate rate based dynamics of neuronal circuits composed of several subpopulations and random connectivity. Nonzero connections are either positive-for excitatory neurons, or negative for inhibitory ones, while single neuron output is strictly positive; in line with known constraints in many biological systems. Using Dynamic Mean Field Theory, we find the phase diagram depicting the regimes of stable fixed point, unstable dynamic and chaotic rate fluctuations. We characterize the properties of systems near the chaotic transition and show that dilute excitatory-inhibitory architectures exhibit the same onset to chaos as a network with Gaussian connectivity. Interestingly, the critical properties near transition depend on the shape of the single- neuron input-output transfer function near firing threshold. Finally, we investigate network models with spiking dynamics. When synaptic time constants are slow relative to the mean inverse firing rates, the network undergoes a sharp transition from fast spiking fluctuations and static firing rates to a state with slow chaotic rate fluctuations. When the synaptic time constants are finite, the transition becomes smooth and obeys scaling properties, similar to crossover phenomena in statistical mechanics

Subfields: Cognitive Neuroscience, Electrical and Electronic Engineering, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/69783104781ec61316e1d3cb56483a9c1cf8607b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

1. Boris Hanin
2. Greg Yang

In Electrical and Electronic Engineering

1. Michael Backes
2. Greg Yang

In Statistical and Nonlinear Physics

1. Boris Hanin
2. David Rolnick
3. Michael Backes
4. Greg Yang
5. Steven Bethard
6. Samuel R. Bowman

Citations in the corpus

• How to Start Training: The Effect of Initialization and Architecture (2018, background): “[8] already mentioned above, as well as [14, 22, 23, 25].”
• Shared cortex-cerebellum dynamics in the execution and learning of a motor task (2019, unknown): “First, dynamics and correlations will vary by the timescale of analysis (Cohen and Kohn, 2011; Kadmon and Sompolinsky, 2015), which in our data are set by the relatively slow GCaMP kinetics.”
• On the security relevance of weights in deep learning. (2020, background): “State of the art approaches rely on the idea that given a random initialization, the variance of weights is particularly important [10, 11] and determines the dynamics of the networks [15, 30].”
• Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation (2019, background): “Recently, Lee et al. (2018); Matthews et al. (2018) empirically and/or theoretically investigated analogous correspondences for infinite width, finite depth deep MLPs, and likewise Novak et al.”
• Linking Connectivity, Dynamics, and Computations in Low-Rank Recurrent Neural Networks. (2017, unknown): “As in classical DMF derivations (Sompolinsky et al., 1988; Rajan et al., 2010; Kadmon and Sompolinsky, 2015), x1, x2 and y are standard normal variables.”
• Adversarial Initialization - when your network performs the way I want (2019, background): “State of the art approaches rely on the idea that given a random initialization, the variance of weights is particularly important [11], [12] and determines the dynamics of the networks [16], [30].”