The Enhanced Storage Capacity in Neural Networks with Low Activity Level

By Misha Tsodyks, M. V. Feigel'man. 1988

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Structural Synaptic Plasticity Has High Memory Capacity and Can Explain Graded Amnesia, Catastrophic Forgetting, and the Spacing Effect (2014, background): “By comparison, even with real-valued synapses that have an infinite number of states, Hebbian-type weight plasticity without structural plasticity achieves less than one bit per synapse [72,73,139,140].”
Re-evaluating circuit mechanisms underlying pattern separation (2019, positive): “As Marr-Albus theory was extended to other brain regions, such ‘ultra-sparse’ population coding formed the core of our current understanding of pattern separation, associative learning, and memory storage (Marr, 1971; Kanerva, 1988; Tsodyks et al., 1988; Rolls and Treves, 1990; Tyrrell and Willshaw,”
Sparseness constrains the prolongation of memory lifetime via synaptic metaplasticity. (2008, background): “In particular, the advantage of metaplasticity may be outweighed by sparse representations of memories (Tsodyks and Feigel’man 1988; Amit and Fusi 1994).”
Associative Data Storage and Retrieval in Neural Networks (1996, background): “In [27] just the same value P = 0:72 bits/syn has been obtained.”
Modeling the spontaneous reactivation of experience‐specific hippocampal cell assembles during sleep (1996, background): “The effect in our simulation occurs because the different environments, and locations within them, are encoded relatively sparsely (Marr, 1971; Tsodyks and Feigelman, 1988; McNaughton, 1989; Amit and Tsodyks, 1991; Gibson and Robinson, 1992).”
Synchronization and pattern separation in spiking associative memories and visual cortical areas (2004, background): “72 for non-binary synapses [189, 134], the bound of ln 2 for the storage capacity normalized to the required physical memory was never exceeded for non-trivial models.”
On the Information Storage Capacity of Local Learning Rules (1992, methodology): “spin-glass models (e.g., Hopfield 1982; Amit ef al. 1987; Horner 1989; Tsodyks and Feigelman 1988 ).”
Neural Associative Memory with Optimal Bayesian Learning (2011, methodology): “…(up to factor η) the linear covariance rule as discussed in much previous work (e.g., Sejnowski, 1977a, 1977b; Hopfield, 1982; Palm, 1988a, 1988b; Tsodyks & Feigel’man, 1988; Willshaw and Dayan, 1990; Dayan & Willshaw, 1991; Palm & Sommer, 1992, 1996; Dayan & Sejnowski, 1993; Chechik et al.,…”
Optimal Plasticity from Matrix Memories: What Goes Up Must Come Down (1990, background): “It is often noted (Willshaw et aZ. 1969; Amit et at. 1987; Gardner 1987; Tsodyks and Feigel'man 1988) that the sparser the coding of information (i.”
Memory Capacities for Synaptic and Structural Plasticity (2010, unknown): “72 (Tsodyks and Feigel’man, 1988; Palm and Sommer, 1992).”