Creating artificial neural networks that generalize

By J. Sietsma, R. Dow. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Scaling & Learning Theory

Citations in the corpus

Adversarial Training Methods for Semi-Supervised Text Classification (2016, background): “There are some previous works adding random noise to the input and hidden layer during training, to prevent overfitting (e.g. (Sietsma & Dow, 1991; Poole et al., 2013)).”
Semi-supervised Learning with Ladder Networks (2015, methodology): “…the Ladder network where the inference process itself can be learned by using the principle of denoising which has been used in supervised learning (Sietsma and Dow, 1991), denoising autoencoders (dAE) (Vincent et al. , 2010) and denoising source separation (DSS) (S¨arel¨a and Valpola, 2005) for…”
Early Stopping without a Validation Set (2017, background): “This test loss, however, is also only an estimator for L (in the same sense as the train loss) with a ﬁnite stochastic error whose variance drops linearly with the test set size.”
On the Transfer of Disentangled Representations in Realistic Settings (2020, background): “Adding noise to the input of neural networks is known to lead to better generalization (Sietsma and Dow, 1991; Bishop, 1995).”
Adversarial Logit Pairing (2018, methodology): “We note that our best results with CLP relied on adding Gaussian noise to the input during training, a standard neural network regularization technique (Sietsma & Dow, 1991).”
Techniques for Learning Binary Stochastic Feedforward Neural Networks (2014, methodology): “Adding noise or stochasticity to the inputs of a deterministic neural network has been found useful as a regularization method (Sietsma and Dow, 1991).”
Optimized neural network using beetle antennae search for predicting the unconfined compressive strength of jet grouting coalcretes (2019, background): “The convergence speed depends on the initialization of the connection weights and bias values.(41) The complexity of the BPNN architecture is dependent on the number of hidden layers.”
Estimation of lignocellulosic biomass pyrolysis product yields using artificial neural networks (2021, background): “As long as the amount of added noise is contained, it can lead to better generalization, since the network becomes less prone to memorization of the data points [57,58].”
Pattern analysis for machine olfaction: a review (2002, methodology): “Finally, training with noise, also known as jitter, can be used to prevent the MLP from approximating the training set too closely and has been shown to improve generalization [59].”
Evolving Unsupervised Deep Neural Networks for Learning Meaningful Representations (2017, background): “Therefore, they have been extensively employed in optimizing NNs, which refers to the discipline of neuroevolution, such as for the connection weight optimization [46–48], the architecture setting [49–51] (more examples can be found in [45]).”
From Variational to Deterministic Autoencoders (2019, background): “Using random noise injection to regularize neural networks during training is a well-known technique that dates back several decades [2, 47].”
Certainty-Driven Consistency Loss for Semi-supervised Learning (2019, methodology): “Based on this assumption, many traditional algorithms have been developed, including Transductive SVM [4, 5], graph-based methods [6, 7], and noise regularization [8, 9, 10].”
Data Augmentation Using Random Image Cropping and Patching for Deep CNNs (2018, background): “Pixel dropping functions as injection of noise into an image [28].”
A Kernel Theory of Modern Data Augmentation (2018, methodology): “For many years it has been used, for example, in the form of jittering and virtual examples in the neural network and kernel methods literatures [35, 34, 8].”
Does Unsupervised Architecture Representation Learning Help Neural Architecture Search? (2020, methodology): “Using random noise injection mechanism has been proved to be effective on the regularization of neural networks [49, 50, 36].”
Learning with Marginalized Corrupted Features (2013, methodology): “Further, MCF could be used in the training of neural networks with a single layer of hidden units: blankout noise on the hidden nodes can improve the performance of the networks (LeCun et al., 1990; Sietsma & Dow, 1991; Hinton et al., 2012) and can be marginalized out analytically.”