ROBUST TIMING AND MOTOR PATTERNS BY TAMING CHAOS IN RECURRENT NEURAL NETWORKS

By Rodrigo Laje, Dean V. Buonomano. Nature Neuroscience, 2012

455 citations recorded by the source corpus.

The brain's ability to tell time and produce complex spatiotemporal motor patterns is critical for anticipating the next ring of a telephone or playing a musical instrument. One class of models proposes that these abilities emerge from dynamically changing patterns of neural activity generated in recurrent neural networks. However, the relevant dynamic regimes of recurrent networks are highly sensitive to noise; that is, chaotic. We developed a firing rate model that tells time on the order of seconds and generates complex spatiotemporal patterns in the presence of high levels of noise. This is achieved through the tuning of the recurrent connections. The network operates in a dynamic regime that exhibits coexisting chaotic and locally stable trajectories. These stable patterns function as 'dynamic attractors' and provide a feature that is characteristic of biological systems: the ability to 'return' to the pattern being generated in the face of perturbations.

Subfields: Artificial Intelligence, Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/cab6de30b60f50c0da4d30063271acab69ff20c9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

Citations in the corpus

At the Edge of Chaos: How Cerebellar Granular Layer Network Dynamics Can Provide the Basis for Temporal Filters (2015, background): “Theoretical investigations of RNNs, both from within the reservoir-computing framework [23–25] and outside it [26,29,30,68,69] suggest that, in very broad terms, the outputs of networks with weak recurrent connections tend to reproduce the temporal structure of their input, whereas when the recurren”
The Neural Basis of Timing: Distributed Mechanisms for Diverse Functions (2018, background): “Specifically, networks with strong recurrent connections are capable of generating continuously changing patterns of self-perpetuating activity, and when the recurrent weights are appropriately tune the resulting neural trajectories can robustly encode time (Laje and Buonomano, 2013).”
Controlling Recurrent Neural Networks by Conceptors (2014, background): “This idea of “internalizing” a driven dynamics into a reservoir has been independently (re)introduced under different names and for a variety of purposes (self-prediction [72], equilibration [55], reservoir regularization [90], self-sensing networks [100], innate training [61]) and appears to be a f”
Using Conceptors to Manage Neural Long-Term Memories for Temporal Patterns (2017, background): “In machine learning, RNNs trained on sequence prediction/generation tasks are typically considered as models of a stochastic process, for instance the process whose realizations are Wikepedia texts, as in Sutskever et al. (2011), rather than as cases of neural memory.”
Why neurons mix: high dimensionality for higher cognition. (2016, background): “Moreover, high-dimensional neural representations are also a crucial component of dynamical models of recurrent neural networks like echo state machines or liquid state machines [35–40].”
Physical reservoir computing—an introductory perspective (2020, background): “Chaos is more apparently exploited in the learning scheme, which is called innate training [146].”
Low-dimensional dynamics for working memory and time encoding (2020, methodology): “In contrast, the cumulative dimensionality increases linearly in the stabilized random network (17).”
How the motor system both encodes and influences our sense of time (2016, background): “It seems that the motor system routinely generates strong dynamic signals in order to internally represent time, predict sensory events, and drive behaviour [17].”
Reward-based training of recurrent neural networks for cognitive and value-based tasks (2016, methodology): “…working memory (Mante et al., 2013; Barak et al., 2013; Carnevale et al., 2015; Song et al., 2016; Miconi, 2016), timing and sequence generation (Laje & Buonomano, 2013; Rajan et al., 2015), and motor reach (Hennequin et al., 2014; Sussillo et al., 2015), can reproduce important features of…”
Recurrent Network Models of Sequence Generation and Memory. (2016, methodology): “A compelling example, FORCE learning (Sussillo and Abbott, 2009) has been implemented as part of several modeling studies (Laje and Buonomano, 2013; Sussillo, 2014 and references therein), but it had a few limitations.”
Attractor and integrator networks in the brain (2021, background): “Robust sequences can be constructed as low-dimensional limit cycle attractors, in which high-dimensional perturbations are corrected, while along the attractor there is a systematic, periodic, or quasiperiodic flow of states [70, 71, 72, 73, 74].”
Flexible sensorimotor computations through rapid reconfiguration of cortical dynamics (2018, unknown): “This line of investigation has been complemented by efforts in developing, training, and analyzing recurrent neural network models that can emulate a range of motor and cognitive behaviors, leading to novel insights into the underlying latent dynamics (Rigotti et al., 2010; Laje and Buonomano, 2013;”
Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules (2016, background): “How learning of the recurrent connections (Laje and Buonomano, 2013; Hennequin et al., 2014) can be achieved by STDP under neuromodulation, while maintaining sustained balanced network activity (van Vreeswijk and Sompolinsky, 1996; Brunel, 2000; Vogels et al., 2005) remains an open question.”