Nonlinear Mixup: Out-Of-Manifold Data Augmentation for Text Classification

By Hongyu Guo. 2020

125 citations recorded by the source corpus.

Data augmentation with Mixup (Zhang et al. 2018) has shown to be an effective model regularizer for current art deep classification networks. It generates out-of-manifold samples through linearly interpolating inputs and their corresponding labels of random sample pairs. Despite its great successes, Mixup requires convex combination of the inputs as well as the modeling targets of a sample pair, thus significantly limits the space of its synthetic samples and consequently its regularization effect. To cope with this limitation, we propose “nonlinear Mixup”. Unlike Mixup where the input and label pairs share the same, linear, scalar mixing policy, our approach embraces nonlinear interpolation policy for both the input and label pairs, where the mixing policy for the labels is adaptively learned based on the mixed input. Experiments on benchmark sentence classification datasets indicate that our approach significantly improves upon Mixup. Our empirical studies also show that the out-of-manifold samples generated by our strategy encourage training samples in each class to form a tight representation cluster that is far from others.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

A Survey of Data Augmentation Approaches for NLP (2021, background): “, 2020), more general mixing schemes (Guo, 2020), and adding adversaries (Beckham et al.”
i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning (2021, background): “…hand, MixUp has also shown its effectiveness in other tasks and non-vision domains, including generative adversarial networks (Lucas et al., 2018), improved robustness and uncertainty (Hendrycks et al., 2020), and sentence classiﬁcation in natural language processing (Guo, 2020; Guo et al., 2019).”
A Survey on Data Augmentation for Text Classification (2021, unknown): “Guo [140] further advances the wordMixup approach by using a nonlinear interpolation policy.”