ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learning

By Viktor Olsson, Wilhelm Tranheden, Juliano Pinto, Lennart Svensson. IEEE Workshop/Winter Conference on Applications of Computer Vision, 2020

420 citations recorded by the source corpus.

The state of the art in semantic segmentation is steadily increasing in performance, resulting in more precise and reliable segmentations in many different applications. However, progress is limited by the cost of generating labels for training, which sometimes requires hours of manual labor for a single image. Because of this, semi-supervised methods have been applied to this task, with varying degrees of success. A key challenge is that common augmentations used in semi-supervised classification are less effective for semantic segmentation. We propose a novel data augmentation mechanism called ClassMix, which generates augmentations by mixing unlabelled samples, by leveraging on the network’s predictions for respecting object boundaries. We evaluate this augmentation technique on two common semi-supervised semantic segmentation benchmarks, showing that it attains state-of-the-art results. Lastly, we also provide extensive ablation studies comparing different design decisions and training regimes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/6d78ad8f84d486a27c6d1501803408fbfa276bd6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels (2022, methodology): “Several methods [17, 44, 45] utilize strong data augmentation such as CutOut [15], CutMix [46], and ClassMix [32] based on self-training.”
DACS: Domain Adaptation via Cross-domain Mixed Sampling (2020, methodology): “In SSL the perturbations have typically been based on image augmentations [6, 9, 36, 37, 38, 39].”
Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation (2021, background): “Image mixing is popular for classification [93], [94], [95], [96], [97], [98] yet, to the best of our knowledge, sees limited use for semantic segmentation [99], [100], [101], [102], [103], and has never been considered to design memory-efficient rehearsal learning systems.”
Bootstrapping Semantic Segmentation with Regional Contrast (2021, methodology): “We apply the Mean Teacher framework (Tarvainen & Valpola, 2017) following prior state-of-the-art semi-supervised segmentation methods (Olsson et al., 2021; Mittal et al., 2019).”
Self-Training: A Survey (2022, positive): “More recently, [Olsson et al., 2021] advocated using network predictions as a guidance to retain object boundaries during the mixing phase of image segmentation.”
DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation (2021, methodology): “ST has been shown to be particularly efficient if the student network gθ is trained on augmented target data, while the teacher network hφ generates the pseudo-labels using non-augmented target data for semi-supervised learning [16, 61, 65] and unsupervised domain adaptation [1, 67] In this work, we”
Give Me Your Trained Model: Domain Adaptive Semantic Segmentation without Source Data (2021, methodology): “Particularly, we divide the target domain into two parts and employ a semi-supervised learning technique, ClassMix [21], to boost the adaptation performance, which acts as a generic simple and effective post-processing module for domain adaptation.”
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization (2021, background): “Related ideas have been applied to semi-supervised semantic segmentation [40] and tailored data augmentation strategies have been explored [22, 68].”