The Role of Constraints in Hebbian Learning

By Kenneth D. Miller, David John Cameron MacKay. 1994

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Continual Lifelong Learning with Neural Networks: A Review (2018, background): “However, similar to most of the abovedescribed approaches, the use of pseudo-rehearsal methods was strictly evaluated on two datasets of relatively low complexity, e.g. the MNIST and the Street View House Number (SVHN) (Netzer et al., 2011).”
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “, 1982) and various constraints (Miller and Mackay, 1994), or more advanced update dynamics such as dual weights for fast and slow decay (Levy and Bairaktaris, 1995; Hinton and Plaut, 1987; Bullinaria, 2009a; Soltoggio, 2015).”
The temporal paradox of Hebbian learning and homeostatic plasticity (2017, methodology): “In theoretical models, competition and stability are often achieved by augmenting Hebbian plasticity with additional constraints [3, 5, 7].”
Generalization in Interactive Networks: The Benefits of Inhibitory Competition and Hebbian Learning (2001, unknown): “…instead of the multiplicative normalization used in the competitive learning rule, because subtractive normalization moves the weights towards extrema, while multiplicative normalization retains a linear proportionality to the correlation strength (Miller & MacKay, 1994; Goodhill & Barrow, 1994).”
Synaptic Plasticity in Neural Networks Needs Homeostasis with a Fast Rate Detector (2013, background): “they require cell wide spatial averaging over synapses, which can only be achieved in a plausible way if all synaptic weights decay at a global rate modulated by the total afferent synaptic strength [16].”
Homeostatic plasticity in the developing nervous system (2004, background): “From virtually the first time computational modellers explored the properties of Hebbian learning rules, it became apparent that selective changes in synaptic weights are difficult to achieve without building in some homeostatic or 'normalizing' mechanism to regulate total synaptic strength or excit”
Firing Rate Homeostasis in Visual Cortex of Freely Behaving Rodents (2013, background): “…to this instability problem is to endow neurons with homeostatic plasticity mechanisms that keep neuronal firing rates within a set point range (Miller and MacKay, 1994), but whether neuronal firing in the intact CNS is homeostatically regulated remains a critical and untested prediction of…”
Competitive Hebbian learning through spike-timing-dependent synaptic plasticity (2000, methodology): “The constraints used in many models of Hebbian learnin”
Modeling the dynamic interaction of Hebbian and homeostatic plasticity. (2014, background): “This is a positive feedback process— strong synapses grow stronger—that in models typically leads to synaptic instability in the absence of additional biological constraints (Miller and MacKay, 1994; Turrigiano, 2008).”
Synaptic plasticity: taming the beast (2000, background): “A frequent approach in neural network models is to globally adjust all the synapses onto each postsynaptic neuron based on its level of activit”
SPIKING NEURON MODELS Single Neurons , Populations , Plasticity (2002, background): “Practically, competition can be implemented in simulations by normalizing the sum of all weights converging onto the same postsynaptic neuron (Miller and MacKay, 1994); cf.”
Heterosynaptic Plasticity Prevents Runaway Synaptic Dynamics (2013, background): “Mechanisms supporting runaway dynamics of synaptic weights combined with strong synaptic competition (Miller and MacKay 1993; Miller, 1996) are useful for formation of sensory representations (Wiesel and Hubel, 1963; Aitkin et al.”
Trusted Machine Learning for Probabilistic Models (2016, background): “In the ML literature, there is a rich history of learning under constraints (Dietterich, 1985; Miller & MacKay, 1994).”
Partial Breakdown of Input Specificity of STDP at Individual Synapses Promotes New Learning (2016, methodology): “The need for fast-scale mechanism(s) which prevent runaway dynamics of activity and support synaptic competition has been long appreciated in theoretical and modeling studies (von der Malsburg, 1973; Oja, 1982; K. M. Miller, 1994; K. D. Miller, 1996).”
Hebbian plasticity requires compensatory processes on multiple timescales (2017, methodology): “Second, it is compensatory because it ensures stability by effectively enforcing a constraint on the afferent synaptic weights [10,57].”