Opening the Black Box: Low-Dimensional Dynamics in High-Dimensional Recurrent Neural Networks

By David Sussillo, Omri Barak. Neural Computation, 2013

562 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/5dd52311ccce1a57473b55b0d36cbd40bf0f59f0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Capacity and Trainability in Recurrent Neural Networks (2016, methodology): “Further, Sussillo & Barak (2013) have reverse-engineered the vanilla RNN trained on simple tasks, using the tools and language of nonlinear dynamical systems theory.”
Motor primitives in space and time via targeted gain modulation in cortical networks (2018, unknown): “Although this is an effective way of mimicking gain changes in recurrent network models with strongly nonlinear single-neuron dynamics [37, 43], we were unable to produce desired target outputs by training a tonic input.”
Reward-based training of recurrent neural networks for cognitive and value-based tasks (2016, methodology): “However, methods for revealing neural mechanisms in such networks remain limited to state-space analysis (Sussillo and Barak, 2013), which in particular does not reveal how the synaptic connectivity leads to the dynamics responsible for implementing the higher-level decision policy.”
Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future (2020, background): “…studies in computational neuroscience have trained simple recurrent neural networks to perform tasks and then interrogated the properties of the trained networks for clues as to how they work (Barak, 2017; Foerster, Gilmer, Sohl-Dickstein, Chorowski, & Sussillo, 2017; Sussillo & Barak, 2013).”
Manifold GPLVMs for discovering non-Euclidean latent structure in neural data (2020, methodology): “Our framework could be used to dissect such representations, adding to a growing toolbox for the analysis of artiﬁcial neural networks (Sussillo and Barak, 2013).”
Organizing recurrent network dynamics by task-computation to enable continual learning (2020, methodology): “This is done by analyzing linearized dynamics around ﬁxed points of the network for each task-period [28–30].”
Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity (2021, background): “Comparison of neural network models to brain activity can narrow the space of hypotheses [50], and detailed analysis of candidate models can give insight into the underlying computational mechanisms [79].”
Training Excitatory-Inhibitory Recurrent Neural Networks for Cognitive Tasks: A Simple and Flexible Framework (2016, background): “In this regard, although state-space analyses of fixed and “slow” points [83] are illuminating they do not yet explain how the network’s connectivity, combined with the nonlinear activation functions, lead to the observed neural trajectories.”
Input Switched Affine Networks: An RNN Architecture Designed for Interpretability (2016, methodology): “For example, Sussillo & Barak (2013) used linearization and nonlinear dynamical systems theory to understand RNNs solving a set of simple but varied tasks.”
Artificial Intelligence in Surgery: Promises and Perils (2018, background): “52 Although the automated nature of neural recognize that the accountability of algorithms, the safety/verifiability of automated analyses, and the implications of these analyses on human machine interactions can impact the utility of AI in clinical practice.”
Artificial Neural Networks for Neuroscientists: A Primer (2020, background): “First, we should continue to bring RNNs closer to neurobiology.”
Motor cortex embeds muscle-like commands in an untangled population response (2018, unknown): “…body of work seeks to understand neural responses in terms of signals that a recurrent or feedback-driven neural network would need to perform the relevant task (Hennequin et al., 2014; Li et al., 2016; Lillicrap and Scott, 2013; Mante et al., 2013; Michaels et al., 2016; Sussillo and Barak, 2013).”
Unsupervised Discovery of Demixed, Low-Dimensional Neural Dynamics across Multiple Timescales through Tensor Component Analysis (2017, background): “Notably, while previous studies have focused on reverse-engineering RNNs with static parameters [51], few works have attempted to characterize how computational mechanisms in RNNs emerge over the process of learning, or optimization, of network parameters.”
Complementary roles of dimensionality and population structure in neural computations (2020, background): “From the neural state-space perspective, the collective dynamics explore a low-dimensional recurrent subspace, and the sub-population structure shapes the non-linear dynamical landscape of the activity in this subspace [Sussillo and Barak, 2013].”
Computation Through Neural Population Dynamics (2020, background): “The fixed points and corresponding linearizations of an RNN’s dynamics can be identified via numerical optimizations (Golub & Sussillo 2018, Sussillo & Barak 2013).”
Linking Connectivity, Dynamics, and Computations in Low-Rank Recurrent Neural Networks. (2017, unknown): “While more work is needed to establish under which precise conditions a low-rank network provides a good computational approximation of a full recurrent network, this conjecture provides a simple and practically useful working hypothesis for reverse-engineering trained neural networks (Sussillo and ”
Controlling Recurrent Neural Networks by Conceptors (2014, background): “This idea can be traced back at least to the notion of cell assemblies formulated by Hebb [45], reached a first culmination in the formal analysis of associative memories [80, 48, 4], and has since then diversified into a range of increasingly complex models of interacting (partial) neural attractor”