Semi-supervised semantic segmentation needs strong, varied perturbations

By Geoff French, Samuli Laine, Timo Aila, Michal Mackiewicz, Graham David Finlayson. British Machine Vision Conference, 2019

378 citations recorded by the source corpus.

Consistency regularization describes a class of approaches that have yielded ground breaking results in semi-supervised classification problems. Prior work has established the cluster assumption - under which the data distribution consists of uniform class clusters of samples separated by low density regions - as important to its success. We analyze the problem of semantic segmentation and find that its' distribution does not exhibit low density regions separating classes and offer this as an explanation for why semi-supervised segmentation is a challenging problem, with only a few reports of success. We then identify choice of augmentation as key to obtaining reliable performance without such low-density regions. We find that adapted variants of the recently proposed CutOut and CutMix augmentation techniques yield state-of-the-art semi-supervised semantic segmentation results in standard datasets. Furthermore, given its challenging nature we propose that semantic segmentation acts as an effective acid test for evaluating semi-supervised regularizers. Implementation at: this https URL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/c2dfccb7e3fe860e3e3dd1658ace1e1a6a2edef9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization (2021, background): “Related ideas have been applied to semi-supervised semantic segmentation [40] and tailored data augmentation strategies have been explored [22, 68].”
HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation (2022, neutral): “Therefore, several methods aim to reduce the annotation burden such as weakly-supervised learning [14, 61, 93], semi-supervised learning [17, 27, 35, 53], and unsupervised domain adaption (UDA) [26, 28, 59, 67, 92].”
Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels (2022, methodology): “To alleviate this problem, many attempts [1,4,11,17,23,34,44,49] have been made towards semi-supervised semantic segmentation, which learns a model with only a few labeled samples and numerous unlabeled ones.”
Milking CowMask for Semi-Supervised Image Classification (2020, methodology): “In this paper, we use the CowMask (French et al., 2019) masking strategy, recently introduced for semi-supervised semantic segmentation, instead of rectangular masking, in semi-supervised image classiﬁcation tasks.”
MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation (2022, methodology): “It is a common strategy used in semi-supervised learning [17,29,71] and UDA [1,30,31,72].”
Self-Training: A Survey (2022, neutral): “Other studies refined the Mean-Teacher approach with a data-augmentation technique by combining two images with random patches to improve prediction consistency [French et al., 2020; Xie et al., 2020a].”
ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learning (2020, methodology): “This results in a value between 0 and 1, which we empirically found to serve as an adequate proxy for the quality of the predictions, roughly in line with [13, 14].”
DACS: Domain Adaptation via Cross-domain Mixed Sampling (2020, methodology): “Our method is inspired by recently proposed methods of using mixed samples for SSL [11, 17, 29], and we show that applying the naive implementation of these methods directly to UDA causes classes to become conflated, i.”