A spiking network model of short-term active memory

By David Zipser, B. Kehoe, Gwen Littlewort, Joaquín M. Fuster. 1993

Studies of cortical neurons in monkeys performing short-term memory tasks have shown that information about a stimulus can be maintained by persistent neuron firing for periods of many seconds after removal of the stimulus. The mechanism by which this sustained activity is initiated and maintained is unknown. In this article we present a spiking neural network model of short-term memory and use it to investigate the hypothesis that recurrent, or “re-entrant,” networks with constant connection strengths are sufficient to store graded information temporarily. The synaptic weights that enable the network to mimic the input-output characteristics of an active memory module are computed using an optimization procedure for recurrent networks with non-spiking neurons. This network is then transformed into one with spiking neurons by interpreting the continuous output values of the nonspiking model neurons as spiking probabilities. The behavior of the model neurons in this spiking network is compared with that of 179 single units previously recorded in monkey inferotemporal (IT) cortex during the performance of a short-term memory task. The spiking patterns of almost every model neuron are found to resemble closely those of IT neurons. About 40% of the IT neuron firing patterns are also found to be of the same types as those of model neurons. A property of the spiking model is that the neurons cannot maintain precise graded activity levels indefinitely, but eventually relax to one of a few constant activities called fixed-point attractors. The noise introduced into the model by the randomness of spiking causes the network to jump between these attractors. This switching between attractor states generates spike trains with a characteristic statistical temporal structure. We found evidence for the same kind of structure in the spike trains from about half of the IT neurons in our test set. These results show that the behavior of many real cortical memory neurons is consistent with an active storage mechanism based on recurrent activity in networks with fixed synaptic strengths.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, methodology): “Many computational models of such spiking neurons have been proposed and analyzed (e.g., Gerstner and van Hemmen, 1992; Zipser et al., 1993; Stemmler, 1996; Tsodyks et al., 1996; Maex and Orban, 1996; Maass, 1996, 1997; Kistler et al., 1997; Amit and Brunel, 1997; Kempter et al., 1999; Song et al., ”
Synaptic Basis of Cortical Persistent Activity: the Importance of NMDA Receptors to Working Memory (1999, methodology): “Indeed, most previous model studies used simple firing-rate models (Wilson and Cowan, 1973; Amari, 1977; Zipser et al., 1993; Amit et al., 1994; Camperi and Wang, 1998; Moody et al., 1998).”
The prefrontal cortex—an update: Time of the essence (2001, unknown): “Cells in posterior cortex of sensory association show working-memory sample-and-hold operations as delay tasks require, Zipser et al. (1993) observed " cells " (hidden units) that properties for stimuli of the specific modality they process .”
Multilevel structure in behaviour and in the brain: a model of Fuster's hierarchy (2007, background): “function (Zipser 1991; Zipser et al. 1993; Moody et al. 1998).”
Interactions between frontal cortex and basal ganglia in working memory: A computational model (2001, background): “Perhaps the most obvious means of achieving the kinds of actively maintained neural firing observed in PFC neurons using basic neural mechanisms is to have recurrent excitationamong frontal neurons, resulting in attractor states that persist over time (e.g., Braver & Cohen, 2000; Dehaene & Changeux,”
Low-dimensional dynamics for working memory and time encoding (2020, methodology): “This is significant because neural connectivity in the artificial RNN was initialized randomly before training, and unit activity was not constrained to replicate neural data during training; the artificial RNN was only told “what” to do but not “how” it should be done (8, 27, 29).”
Synaptic mechanisms and network dynamics underlying spatial working memory in a cortical network model. (2000, methodology): “Most previous computational analyses have used firing rate models (Wilson and Cowan, 1973; Amari, 1977; Zipser et al., 1993; Amit et al., 1994; Seung, 1996; Camperi and Wang, 1998; Moody et al., 1998; Durstewitz et al., 1999).”