ReMixMatch: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring

By David Berthelot, Nicholas Carlini, Ekin Dogus Cubuk, Alexey Kurakin, Kihyuk Sohn, Colin Raffel. 2020

479 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning (2020, background): “In the semi-supervised setting [55, 56], an unsupervised loss is combined with a classification loss over a handful of labels to ground the training [19, 20, 57, 58, 59, 60, 61, 62].”
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence (2020, methodology): “Inspired by UDA [54] and ReMixMatch [3], we leverage Cutout [14], CTAugment [3], and RandAugment [11] for strong augmentation, which all produce heavily-distorted versions of a given image.”
Interpolation Consistency Training for Semi-Supervised Learning (2019, methodology): “, 2019), the methods in Berthelot et al. 2020), Berthelot, Carlini et al. (2019) and Sohn et al. (2020) ave achieved state-of-the-art experimental results on benchark datasets and architectures. Similar to Verma et al. (2019), erthelot et al.”
Image Augmentations for GAN Training (2020, background): “More recently, it also becomes one of the most impetus on semi-supervised learning and unsupervised learning [37, 2, 34, 38, 3, 7].”
CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning (2021, methodology): “In addition, we extend CReST to CReST+ by incorporating distribution alignment [1] with a temperature scaling factor to control its alignment strength over generations, so that predicted data distributions are more aggressively adjusted to alleviate model bias.”
FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling (2021, unknown): “ReMixMatch [13] also leveraged such strong augmentations.”
OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in an Open World (2020, methodology): “Transfer Learning [17, 21, 25] aims to transfer the knowledge of a labeled dataset to another dataset.”
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization (2021, methodology): “The literature on SSL is vast and some classical SSL techniques include pseudolabeling [50, 2, 8, 83], consistency regularization [80, 49, 87, 23, 83], and various data augmentation techniques [7, 6, 91] (also see Sec.”
A Simple Semi-Supervised Learning Framework for Object Detection (2020, methodology): “Lately, complex data augmentation strategies, such as RandAugment [7] or CTAugment [3], have turned out to be powerful for SSL on image classification [58,3,49,59].”
Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning (2020, methodology): “Various perturbation operations have been explored, including basic image processing operators (Xie et al. 2019), dedicated operators such as MixUp (Zhang et al. 2018; Berthelot et al. 2020), learned transformations (Cubuk et al. 2018), and adversarial perturbations (Miyato et al. 2018).”
PseudoSeg: Designing Pseudo Labels for Semantic Segmentation (2020, methodology): “Successful examples include MeanTeacher (Tarvainen & Valpola, 2017), UDA (Xie et al., 2019), MixMatch (Berthelot et al., 2019b), FeatMatch (Kuo et al., 2020), and FixMatch (Sohn et al., 2020a).”
A Survey on Deep Semi-Supervised Learning (2021, methodology): “, RandAugment [103] and CTAugment [113].”
Self-supervised Label Augmentation via Input Transformations (2019, background): “We think that our work could bring in many interesting directions for future research; for instance, one can revisit prior works on applications of self-supervision, e.g., semi-supervised learning with selfsupervision (Zhai et al., 2019; Berthelot et al., 2020).”
In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning (2021, methodology): “Several SSL works combine ideas from both consistency regularization and pseudo-labeling (Berthelot et al., 2019; 2020; Zhou et al., 2020).”
Dash: Semi-Supervised Learning with Dynamic Thresholding (2021, methodology): “Speciﬁcally, SOTA baselines are MixMatch [Berthelot et al., 2019b], UDA [Xie et al., 2020a], ReMixMatch [Berthelot et al., 2019a], FixMatch [Sohn et al., 2020] and the algorithm RYS from [Ren et al., 2020] 3 .”