Noise injection into inputs in back-propagation learning

By Kiyotoshi Matsuoka. 1992

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Dropout Training as Adaptive Regularization (2013, background): “There is a well-known connection between artificial feature corruption and regularization [8, 9, 10].”
Deep learning in neural networks: An overview (2014, neutral): “Compare perturbation tolerance conditions (Bishop, 1993; Carter, Rudolph, & Nucci, 1990; Hanson, 1990; Kerlirzin & Vallet, 1993; Matsuoka, 1992; Minai & Williams, 1994; Murray & Edwards, 1993; Neti, Schneider, & Young, 1992).”
Reliable Uncertainty Estimates in Deep Neural Networks using Noise Contrastive Priors (2018, methodology): “Input and output regularization There is classic work on adding noise to inputs for improved generalization (Matsuoka, 1992; An, 1996; Bishop, 1995).”
Flat Minima (1997, unknown): “…8 ij : v ij = w ij g. Flatness condition 1 is a \robustness condition" (or \fault tolerance condition", or \perturbation tolerance condition" { see, e.g., Minai & Williams, 1994; Murray & Edwards, 1993; Neti et al., 1992; Matsuoka, 1992; Bishop, 1993; Kerlirzin & Vallet, 1993; Carter et al., 1990).”
Noise Contrastive Priors for Functional Uncertainty (2018, methodology): “We set µy = y which corresponds to data augmentation (Matsuoka, 1992; An, 1996), where a model is trained to recover the true labels from perturbed inputs.”
Feature Noising for Log-Linear Structured Prediction (2013, background): “Sometimes, training with corrupted features reduces to a special form of regularization (Matsuoka, 1992; Bishop, 1995; Rifai et al., 2011b; Wager et al., 2013).”
Neural networks in geophysical applications (2000, methodology): “Two proposed techniques that do not have these drawbacks are the incremental algorithms of Moody (1994) and Jutten and Chentouf (1995).”
Model Regularization for Stable Sample Rollouts (2014, background): “There is a long history in the supervised learning setting of adding noise to training data (or otherwise intentionally corrupting it) to obtain regularization and generalization benefits (see e.g. Matsuoka 1992, Burges & Schölkopf 1997, Maaten et al. 2013).”
Two Distributed-State Models For Generating High-Dimensional Time Series (2011, unknown): “Different models will have different partition functions.”
Fast dropout training (2013, background): “Fast dropout fits into the general framework of integrating out noise added to the training data (Matsuoka, 1992; Bishop, 1995).”
Synthetic and Natural Noise Both Break Neural Machine Translation (2017, background): “Training with noise can provide a form of regularization (Bishop, 1995) and ensure the model is exposed to samples outside the training distribution (Matsuoka, 1992).”
The Effects of Adding Noise During Backpropagation Training on a Generalization Performance (1996, methodology): “(1992) and Matsuoka (1992) previously obtained the PI term of equation 3.24.”