Vicinal Risk Minimization

By Olivier Chapelle, Jason Aaron Edward Weston, L. Eon Bottou, Vladimir Naumovich Vapnik. 2000

392 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

mixup: Beyond Empirical Risk Minimization (2017, methodology): “, 1998), formalized by the Vicinal Risk Minimization (VRM) principle (Chapelle et al., 2000).”
A Survey of Data Augmentation Approaches for NLP (2021, background): “, 2018), interpolates the inputs and labels of two or more real examples, a paradigm with theoretical underpinnings (Chapelle et al., 2001).”
Distributional Smoothing with Virtual Adversarial Training (2015, methodology): “ the local distribution of the inputs.Still another classic way to include the information about the the input distribution is to use local smoothing of the data like Vicinal Risk Minimization (VRM) (Chapelle et al., 2001).VAT, on the other hand, only uses the property of the conditional distributio”
Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, background): “43) to the vicinal risk minimization (VRM) of Chapelle et al. (2001). Specifically, they consider the slightly different regularized functional”
UniformAugment: A Search-free Probabilistic Data Augmentation Approach (2020, background): “Chapelle et al. [1] proposed vicinity risk minimization principle that samples virtual training point out of the vicinity of Gaussian distribution centered at the actual training point.”
DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion (2021, background): “MixUp was shown to have two main effects: (1) it diversifies the training images and thus enlarges the training distribution on the vicinity of each training sample [7] and (2) it improves the network calibration [28, 68], reducing the overconfidence in recent classes.”
Enhanced Convolutional Neural Tangent Kernels (2019, background): “Theoretical study of data augmentation dates back to Chapelle et al. [2001].”
i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning (2021, methodology): “MixUp is a vicinal risk minimization method (Chapelle et al., 2001) that augments data and their labels in a data-driven manner.”
Extracurricular Learning: Knowledge Transfer Beyond Empirical Distribution (2020, background): “, unlabeled data, generative models [43, 58, 31, 22], data augmentation [51, 33, 17], data mixing [67, 59, 66, 26, 21], vicinal distribution [10], etc.”
A Kernel Theory of Modern Data Augmentation (2018, methodology): “For example, previous work (Bishop, 1995; Chapelle et al., 2001) shows that adding noise to input data has the effect of regularizing the model.”
Fairness risk measures (2019, background): “The idea of moving beyond expectations to a general aggregation of the per-instance losses has precedent in learning theory [Chapelle et al., 2001, Maurer and Pontil, 2009] and robust optimisation [Duchi et al., 2016, Gotoh et al., 2018].”
A Comprehensive Survey of Image Augmentation Techniques for Deep Learning (2022, background): “Another natural notion for approximating P ( x, y ) is the vicinity distribution P v ( x, y ), which replaces the Dirac mass function with an estimate of the density in the vicinity of point ( x i , y i ) [79]: where δ v is the vicinity point set of ( x i , y i ) in D .”
Augmenting NLP models using Latent Feature Interpolations (2020, methodology): “One such technique is Mixup - it uses systematic transformations to make sure the model trains on samples from the vicinity distribution (Chapelle et al., 2001) along with the original distribution of the training data.”
SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness (2020, background): “Risk Minimization (VRM) (Olivier Chapelle and Vapnik, 2000) formalizes data augmentation as enlarging the training set support by drawing samples from a vicinity of existing training examples.”
On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks (2019, methodology): “Mixup training [37] is based on the principle of Vicinal Risk Minimization [3](VRM): the classifier is trained not only on the training data, but also in the vicinity of each training sample.”
AdvAug: Robust Adversarial Augmentation for Neural Machine Translation (2020, methodology): “Unlike prior works on vicinal risk minimization (Chapelle et al., 2001; Zhang et al., 2018), we do not directly observe the virtual sentences in Padv or Paut.”
A Robust Minimax Approach to Classification (2002, methodology): “Furthermore, the MPM is related to vicinal risk minimization (Chapelle et al., 2001; Vapnik, 1999), in which SVMs were " improved " using the covariance of the classes to push the hyperplane away from the samples that belong to the class with the largest covariance matrix.”
A Group-Theoretic Framework for Data Augmentation (2019, background): “Starting from [30, 45, 61, 14], a line of research focuses on designing (or learning) neural net architectures that are invariant to certain symmetries [31, 27, 70, 20, 22, 19], equivariant to SO(3) group [21, 28, 29], and others [58, 42, 69, 41].”
Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning (2019, background): “Recently, some data augmentation techniques, such as ﬂipping, rotation or distorting the inputs, follow the learning principle named Vicinal Risk Minimization (VRM) [3] to improve the generalization performance of deep neural networks.”