Current address: Microsoft Research,

Subfields: Natural Language Processing

Citations in the corpus

A Closer Look at Memorization in Deep Networks (2017, background): “While exploring the effect of noise samples on learning dynamics has a long tradition (Bishop, 1995; An, 1996), we are the ﬁrst to examine relationships between the fraction of noise samples and other attributes of the learning algorithm, namely: capacity, training time and dataset size.”
A Survey of Data Augmentation Approaches for NLP (2021, background): “Bishop (1995) show training with noised examples is reducible to Tikhonov regularization (sub-sumes L2).”
Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning (2017, background): “Many classic regularization methods for NNs introduce random perturbation to input or hidden layers [4], [9], [35].”
Synthetic and Natural Noise Both Break Neural Machine Translation (2017, background): “Training with noise can provide a form of regularization (Bishop, 1995) and ensure the model is exposed to samples outside the training distribution (Matsuoka, 1992).”
Deep Learning with Limited Numerical Precision (2015, background): “Moreover, the addition of noise during training has been shown to improve the neural network’s performance (Murray & Edwards, 1994; Bishop, 1995; Audhkhasi et al., 2013).”
Dropout Training as Adaptive Regularization (2013, background): “There is a well-known connection between artiﬁcial feature corruption and regularization [8, 9, 10].”
Adversarial Vulnerability of Neural Networks Increases With Input Dimension (2018, methodology): “This correspondence between training with perturbations and using a regularizer can be compared to Tikhonov regularization: Tikhonov regularization amounts to training with random noise Bishop (1995), while training with adversarial noise amounts to penalizing ∂ x L .”
A Unified Gradient Regularization Family for Adversarial Examples (2015, methodology): “It is worthwhile to note that although it has been long known in [9], that injecting Gaussian noise is equivalent to penalizing the trace of Hessian matrix, to our best knowledge, our approach is the first general method to penalize the gradient of loss function w.”
SmoothGrad: removing noise by adding noise (2017, methodology): “We also ﬁnd that the common regularization technique of adding noise at training time (Bishop, 1995) has an additional “de-noising” effect on sensitivity maps.”
On the Transfer of Disentangled Representations in Realistic Settings (2020, background): “Adding noise to the input of neural networks is known to lead to better generalization (Sietsma and Dow, 1991; Bishop, 1995).”
Reliable Uncertainty Estimates in Deep Neural Networks using Noise Contrastive Priors (2018, background): “Input and output regularization There is classic work on adding noise to inputs for improved generalization (Matsuoka, 1992; An, 1996; Bishop, 1995).”
Robust Convolutional Neural Networks under Adversarial Noise (2015, background): “4; Nguyen et al., 2014). This problem arises because standard CNNs are discriminative models. This work provides a solution to improve instability of CNNs, for example in security applications. While Bishop (1995) showed that training with noise is equivalent to a regularizer, Goodfellow et al. (201”
Do Deep Generative Models Know What They Don't Know? (2018, background): “Moreover, Bishop (1995) showed that training a network under additive Gaussian noise is equivalent to Jacobian regularization, and Rifai et al. (2011) proposed contractive autoencoders , which penalize the Jacobian-norm of the encoder.”
Noisy Networks for Exploration (2017, methodology): “The approach differs from parameter compression schemes such as variational inference [12, 7] and flat minima search [13] since we do not maintain an explicit distribution over weights during training but simply inject noise in the parameters and tune its intensity automatically.”
Stabilizing Training of Generative Adversarial Networks through Regularization (2017, methodology): “i,j g 0 i · g0 j · (@i@jh G) (14) So at the intermediate layer, we would need to effectively operate with a full Hessian, which is computationally demanding, as has already been observed in [5].”
Adversarial Dropout for Supervised and Semi-supervised Learning (2017, methodology): “Several techniques (Hinton et al. 2012; Poole, Sohl-Dickstein, and Ganguli 2014; Bishop 1995b; Lasserre, Bishop, and Minka 2006) have emerged over the past years to address this challenge, and dropout has become one of dominant methods due to its simplicity and effectiveness (Hinton et al. 2012;…”
Inductive biases for deep learning of higher-level cognition (2020, background): “There are many ways to encode such biases—e.g. explicit regularisation objectives (Bishop et al., 1995; Bishop, 1995; Srivastava et al., 2014; Kukačka et al., 2017; Zhang et al., 2017), architectural constraints (Yu and Koltun, 2015; Long et al., 2015; Dumoulin and Visin, 2016; He et al., 2016;…”
Likelihood Ratios for Out-of-Distribution Detection (2019, methodology): “In fact, it has been shown that adding noise to the input is equivalent to adding L 2 regularization to the model weights under some conditions (Bishop, 1995a,b).”