A stochastic version of the delta rule

By Stephen Jose Hanson. 1990

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Weight Agnostic Neural Networks (2019, background): “tolerant. Bayesian Neural Networks The weight parameters of a BNN [4, 5, 27, 47, 74, 84] are not ﬁxed values, but sampled from a distribution. While the parameters of this distribution can be learned [30, 41, 42, 59], the number of parameters is often greater than the number of weights. Recently, Ne”
Deep learning in neural networks: An overview (2014, methodology): “Compare perturbation tolerance conditions (Bishop, 1993; Carter, Rudolph, & Nucci, 1990; Hanson, 1990; Kerlirzin & Vallet, 1993; Matsuoka, 1992; Minai & Williams, 1994; Murray & Edwards, 1993; Neti, Schneider, & Young, 1992).”
Effects of Noise on Convergence and Generalization in Recurrent Networks (1994, background): “G. Tesauro, D. Touretzky,T. Leen, p. 649, MIT Press, 1995. version of the delta rule which adapt weight means and standard deviations insteadof clean weight values (Hanson, 1990).”
An analysis of noise in recurrent neural networks: convergence and generalization (1996, background): “Hanson developed a stochastic version of the delta rule which adapt weight means and standard deviations instead of clean weight values, and demonstrated improved convergence rates [13].”
Linear Transformers Are Secretly Fast Weight Programmers (2021, methodology): “In both settings, we apply 10% dropout (Hanson, 1990; Srivastava et al., 2014), and train using the Adam optimiser (Kingma & Ba, 2014) with an initial learning rate of 0.00025 and 2000 learning rate warm-up steps.”
The Effects of Adding Noise During Backpropagation Training on a Generalization Performance (1996, background): “19911, (2) weight noise that is added to the weights of the neural networks (Hanson 1990; Neuralware 1991; Clay and Sequin 1992; Murray and Edwards 1993; Hinton and van Camp 19931, and (3) Langevin noise that is added to the weight changes (Rognvaldsson 1994).”
Meiosis Networks (1989, background): “Various benchmark tests of this basic algorithm are discussed in Hanson (1990).”
Synaptic Weight Noise During MLP Learning Enhances Fault-Tolerance, Generalization and Learning Trajectory (1992, background): “Much closer to the spirit of this paper is the work of Hanson[Hanson, 90].”