Recurrent Network Models of Sequence Generation and Memory.

By Kanaka Rajan, Christopher D. Harvey, David W. Tank. Neuron, 2016

377 citations recorded by the source corpus.

Sequential activation of neurons is a common feature of network activity during a variety of behaviors, including working memory and decision making. Previous network models for sequences and memory emphasized specialized architectures in which a principled mechanism is pre-wired into their connectivity. Here we demonstrate that, starting from random connectivity and modifying a small fraction of connections, a largely disordered recurrent network can produce sequences and implement working memory efficiently. We use this process, called Partial In-Network Training (PINning), to model and match cellular resolution imaging data from the posterior parietal cortex during a virtual memory-guided two-alternative forced-choice task. Analysis of the connectivity reveals that sequences propagate by the cooperation between recurrent synaptic interactions and external inputs, rather than through feedforward or asymmetric connections. Together our results suggest that neural sequences may emerge through learning from largely unstructured network architectures.

Subfields: Artificial Intelligence, Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/8b67f0d7c2f3ded3e22a200deb98030fc6811477

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

Citations in the corpus

Reward-based training of recurrent neural networks for cognitive and value-based tasks (2016, background): “…et al., 2013; Barak et al., 2013; Carnevale et al., 2015; Song et al., 2016; Miconi, 2016), timing and sequence generation (Laje & Buonomano, 2013; Rajan et al., 2015), and motor reach (Hennequin et al., 2014; Sussillo et al., 2015), can reproduce important features of neural activity recorded in…”
Toward an Integration of Deep Learning and Neuroscience (2016, background): “…the form of the NMDA receptors, which are activated by the simultaneous occurrence of chemical transmitter delivered from the pre-synaptic neuron, and voltage depolarization of the post-synaptic neuron. developed as models of biological working memory and sequence generation (Rajan et al., 2016).”
Towards the neural population doctrine. (2019, methodology): “ANNs have also been used to investigate different tasks including working memory [52,53] and perceptual decision-making [42].”
Flexible sensorimotor computations through rapid reconfiguration of cortical dynamics (2018, positive): “A dynamical systems view has also been used to provide explanations for neural trajectories in premotor and prefrontal cortical areas in various cognitive tasks (Rigotti et al., 2010; Mante et al., 2013; Hennequin et al., 2014; Carnevale et al., 2015; Rajan et al., 2016; Wang et al., 2018).”
Deep Learning With Spiking Neurons: Opportunities and Challenges (2018, background): “Recurrent SNNs have shown remarkable performance in sequence recognition (Zhang et al., 2015; Panda and Srinivasa, 2018) and generation tasks (Rajan et al., 2016; Panda and Roy, 2017).”
Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity (2021, background): “A powerful substrate for instantiating specific hypotheses and generating testable predictions are neural network models that can produce a desired behavior in a given task [51,75] or emulate patterns of neural activity associated with a certain brain area [76–78].”
Organizing recurrent network dynamics by task-computation to enable continual learning (2020, background): “This dynamical systems view has offered insight into computational mechanisms underlying motor control [1], decision making [2–4] and timing tasks [5].”
Closures and Cavities in the Human Connectome (2016, background): “[49] Kanaka Rajan, Christopher D Harvey, and David W Tank.”
Artificial Neural Networks for Neuroscientists: A Primer (2020, background): “In other cases, the target output can directly be neural recording data [McIntosh et al., 2016, Rajan et al., 2016, Andalman et al., 2019].”
Cliques and cavities in the human connectome (2016, background): “2011), memory (Rajan et al. 2016), and behavioral control (Levy et al.”
Complementary roles of dimensionality and population structure in neural computations (2020, background): “Within this viewpoint, neural computations are revealed by studying properties of low-dimensional trajectories of activity in state space [Mante et al., 2013; Rajan et al., 2016; Chaisangmongkon et al., 2017; Remington et al., 2018; Wang et al., 2018; Sohn et al., 2019], while the selectivity of the”