Optimal Plasticity from Matrix Memories: What Goes Up Must Come Down

By David J. Willshaw, Peter S. Dayan. 1990

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, methodology): “Unsupervised learning is implemented in neural networks with self-organizing maps (SOM) (Kohonen, 1982, 1990), auto-encoders (Bourlard and Kamp, 1988), restricted Boltzmann machines (RBM) (Hinton and Salakhutdinov, 2006), Hebbian plasticity (Hebb, 1949; Gerstner and Kistler, 2002a; Cooper, 2005), an”
Computational constraints suggest the need for two distinct input systems to the hippocampal CA3 network (1992, background): “This firing pattern, which constitutes a unitary representation of the episode, results specifically from the direct enthorinal input (Yeckel and Berger, 1990), from the DG-relayed mossy fiber input and, after a brief delay, from the detailed feedback effect of recurrent connections.”
Optimal synaptic learning in non-linear associative memory (2010, background): “Since memory patterns are typically sparse it is most likely that only the synapse usage M11 ∼ BM,pq remains zero.”
Learning and Evolution (1999, background): “Each individual synapse could change its strength according to one of four Hebbian learning rules (Wil lshaw and Dayan, 1990): pure Hebbian, postsynaptic, presynaptic, and covariance.”
Dendritic excitability and synaptic plasticity. (2008, unknown): “However, the inverse of LTP, or LTD, is necessary to optimize information storage in a neural network (476, 688, 840).”
Associative Data Storage and Retrieval in Neural Networks (1996, methodology): “The optimization of the signal-to-noise ratio r carried out by Willshaw and Dayan [37] and independently by Palm [47] has already been suggested { though not carried out { by Hop eld [25].”
Holographic reduced representations (1995, background): “Hetero-associative memories, e.g., holographic memories and matrix memories [37], [8], [22], [5], [38], store a set of pairs of items.”
The relative advantages of sparse versus distributed encoding for associative neuronal networks in the brain (1990, background): “…Levy and Desmond 1985)) In this case to avoid crosstalk upon presentation of pattern 1 one would require and it becomes again possible to find N vectors that satisfy these conditions, provided now that the average activity of these vectors is of the order of U (see also Willshaw and Dayan, 1990).”
Elements of a neurobiological theory of hippocampal function: the role of synaptic plasticity, synaptic tagging and schemas (2006, background): “Heterosynaptic depression may also serve a normalizing function and so increase effective storage capacity (Willshaw & Dayan, 1990).”
Synaptic plasticity and memory: an evaluation of the hypothesis. (2000, background): “Having the capacity to bidirectionally modify synaptic efficacy improves the potential fidelity of memory recall in associative memory matrix models (Willshaw & Dayan 1990).”
Neural Associative Memory with Optimal Bayesian Learning (2011, unknown): “…add linearly (Kohonen, 1972; Kohonen & Oja, 1976; Anderson, Silverstein, Ritz, & Jones, 1977; Hopfield, 1982; Palm, 1988a; 1988b; Tsodyks & Feigel’man, 1988; Willshaw & Dayan, 1990; Dayan & Willshaw, 1991; Palm & Sommer, 1992, 1996; Chechik, Meilijson, & Ruppin, 2001; Sterratt & Willshaw, 2008).”
Revenge of the “Neurds”: Characterizing Creative Thought in Terms of the Structure and Dynamics of Memory (2010, methodology): “One way to do this in neural networks is to use a radial basis function, or RBF (Hancock, Smith, & Phillips, 1991; Holden & Niranjan, 1997; Lu, Sundararajan, & Saratchandran, 1997; Willshaw & Dayan, 1990).”
The Violation of Bell Inequalities in the Macroworld (2000, background): “6 shows schematically how this feature of memory is sometimes modeled in neural networks using a radial basis function (RBF) (Hancock et al., 1991; Holden and Niranjan, 1997; Lu et al. 1997; Willshaw and Dayan, 1990).”
On the Information Storage Capacity of Local Learning Rules (1992, methodology): “Furthermore the local rule achieving maximal signal-to-noise ratio Y cart be identified (see Palm 1990; Willshaw and Dayan 1990) and its value for Y is From this relation one can immediately see that high fidelity, i.e., Y -+ co and nonzero capacity (a --f 0), can be achieved only if p(1 - p ) goes…”
Long-term plasticity of intrinsic excitability: learning rules and mechanisms. (2003, background): “It is well established today that the reversal of LTP preserves a potential for network plasticity and increases the capacity of memory storage (Wilshaw and Dayan 1990).”
Evolving adaptive neural networks with and without adaptive synapses (2003, background): “Floreano and Urzelai chose their rules based on synaptic mechanisms observed in mammals (Willshaw and Dayan 1990).”