Effects of Noise on Convergence and Generalization in Recurrent Networks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Deep learning in neural networks: An overview (2014, background): “Dropout is closely related to older, biologically plausible techniques of adding noise to neurons or synapses during training (e.g., Murray and Edwards, 1993; Schuster, 1992; Nadal and Parga, 1994; Jim et al., 1995; An, 1996), which in turn are closely related to finding perturbation-resistant low-c”
Extraction of rules from discrete-time recurrent neural networks (1996, background): “Jim et al. (1995) show that training noisy recurrent networks with grammatical strings is a good measure of the effect of different types of noise insertion methodologies on networks generalization and convergence performance.”
Accurate deep neural network inference using computational phase-change memory (2019, methodology): “In fact, quantization of the weights or activations 30 , and injecting noise on the weights 31 , activations 32 or gradients 33 have been widely used as DNN regularizers during training to reduce overﬁtting on the training dataset 34,35 .”