Optimal storage properties of neural network models

By E. Gardner, Bernard Derrida. 1988

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Structural Synaptic Plasticity Has High Memory Capacity and Can Explain Graded Amnesia, Catastrophic Forgetting, and the Spacing Effect (2014, background): “First, it is well understood that weight capacity of biologically plausible memory models is limited by hard theoretical bounds suggesting Cwpv0:72 bit per synapse even for an infinite computing precision with N??”
Universal spectrum of normal modes in low-temperature glasses (2015, background): “A standard computation (32, 37) leads to the stability condition.”
Statistical physics of inference: thresholds and algorithms (2015, background): “In physics a case of perceptron that was studied in detail is that of binary synaptic weights xi ∈ {±1} [83, 84, 94].”
Universality of the SAT-UNSAT (jamming) threshold in non-convex continuous constraint satisfaction problems (2017, methodology): “The zero temperature phase diagram of the perceptron has been discussed for σ ≥ 0 in [ 15 ] , and for σ 0 (but | σ | not too large) in [ 16 ] .”
Neural Associative Memory with Optimal Bayesian Learning (2011, background): “On the other hand, there exist general bounds on the storage capacity of neural networks that do not refer to any particular learning algorithm (Gardner, 1988 ; Gardner & Derrida, 1988 ).”
Rademacher complexity and spin glasses: A link between the replica and statistical theories of learning (2019, background): “The link with the VC dimension was discussed in many of these works, notably via its connection with its twin from statistical physics, the Gardner capacity ([9]).”
Statistical mechanics of complex neural systems and high dimensional data (2013, methodology): “achinery designed to understand the statistical mechanics of ﬂuctuating neural activity patterns in the presence of disordered synaptic connectivity matrices. A key conceptual advance made by Gardner [79, 80] was that this same machinery could be applied to the analysis of learning, by performing st”
Entropic gradient descent algorithms and wide flat minima (2020, methodology): “We briefly review how the the geometry of the space of typical Gibbs configurations of the model can be studied using statistical physics techniques [33, 34].”
Optimal information storage and the distribution of synaptic weights: perceptron versus Purkinje cell. (2004, methodology): “By extending a monotonic decrease as a function of synaptic weight, the approach of Gardner and Derrida (1988) and Grini- as predicted here.”