Learning with Marginalized Corrupted Features

By L. V. D. van der Maaten, Minmin Chen, Stephen Tyree, Kilian Q. Weinberger. 2013

175 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Dropout Training as Adaptive Regularization (2013, methodology): “Dropout training falls into the broader category of learning methods that artiﬁcially corrupt training data to stabilize predictions [2, 4, 5, 6, 7].”
To Drop or Not to Drop: Robustness, Consistency and Differential Privacy Properties of Dropout (2015, background): “Interestingly, a recent works by Szegedy et al. [2013] and Maaten et al. [2013] provide a complementary set of experiments: while we study dropout for adversarial removal of the training data, Szegedy et al. [2013] studies adversarial perturbation of test inputs and Maaten et al. [2013] considers…”
Missing Data Imputation for Supervised Learning (2016, methodology): “Future work might compare missing-data perturbation with dropout training, which changes to zero all the values of a random subset of features (Hinton et al., 2012; Maaten et al., 2013; Wang and Manning, 2013).”
Adversarial Dropout for Supervised and Semi-supervised Learning (2017, methodology): “Corrupting training data with noises has been well-known to be a method to stabilize prediction (Bishop 1995a; Maaten et al. 2013; Wager, Wang, and Liang 2013).”
On the Theory and Practice of Privacy-Preserving Bayesian Data Analysis (2016, background): “…of a regularizer, consistent with the noise being interpreted as a “random prior,” and along the lines of noise-based regularization techniques such as (Sri-vastava et al., 2014; van der Maaten et al., 2013), although of course it may correspond to more regularization than we would typically like.”
Deep learning in bioinformatics: introduction, application, and perspective in big data era (2019, background): “The second category acts on the inputs of the model, such as data augmentation and data corruption [96].”
Feature Noising for Log-Linear Structured Prediction (2013, methodology): “Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1170–1179, Seattle, Washington, USA, 18-21 October 2013. c©2013 Association for Computational Linguistics”
Altitude Training: Strong Bounds for Single-Layer Dropout (2014, methodology): “In contrast, by leveraging the conditional independence assumptions of the topic model, we are able to improve the exponent in the rate of convergence of the empirical risk minimizer.”
Steps Toward Deep Kernel Methods from Infinite Neural Networks (2015, background): “Several machine learning theories were devised to explain how deep networks avoid overﬁtting based on dropouts (e.g., [23, 14]).”
DART: Dropouts meet Multiple Additive Regression Trees (2015, methodology): “The technique of dropouts has been used successfully in other learning models (Maaten et al., 2013; Wang and Manning, 2013), for example, in logistic regression (Wager et al., 2013).”
Learning with Pseudo-Ensembles (2014, unknown): “We consider several different measures of variance to penalize, which we will introduce as needed.”
Joint Contrastive Learning with Infinite Possibilities (2020, background): “Among these work, one observation is that variances in feature space along some certain directions essentially correspond to implementing semantic data augmentations in the ambient space [3, 33].”
DisturbLabel: Regularizing CNN on the Loss Layer (2016, background): “Different from other works adding noise to the input unit [36][37][22][23], our work is a way of regularizing a neural network by adding noise to its loss layer (related to the output unit).”
Class-Incremental Learning via Dual Augmentation (2021, methodology): “SemanAug is inspired by MCF [21] and ISDA [22], which have performed semantic augmentation for linear models and DNNs, respectively.”
Model Regularization for Stable Sample Rollouts (2014, background): “There is a long history in the supervised learning setting of adding noise to training data (or otherwise intentionally corrupting it) to obtain regularization and generalization benefits (see e.g. Matsuoka 1992, Burges & Schölkopf 1997, Maaten et al. 2013).”
Transformation Pursuit for Image Classification (2014, background): “In particular, generating virtual examples by adding Gaussian noise directly on the training examples is equivalent to l2-regularization for quadratic loss functions [4]; see [1, 30, 21, 31, 7, 18] for more general equivalence results including drop-out noise [14, 33].”
Invariant representation learning for robust deep networks (2018, unknown): “During training, one can introduce this knowledge stochastically via data augmentation [1], or analytically via a related regularizer [2, 3, 4].”
A Group-Theoretic Framework for Data Augmentation (2019, background): “For example, a line of work proposes to add random or adversarial noise when training [11, 51, 26, 74, 8, 49, 73, 37] and argue that this can lead to a form of regularization [63, 34, 35, 18, 71].”