PseudoSeg: Designing Pseudo Labels for Semantic Segmentation

By Yuliang Zou, Zizhao Zhang, Chun-Liang Li, Xiao Bian, Jia-Bin Huang, Tomas Pfister. International Conference on Learning Representations, 2020

376 citations recorded by the source corpus.

Recent advances in semi-supervised learning (SSL) demonstrate that a combination of consistency regularization and pseudo-labeling can effectively improve image classification accuracy in the low-data regime. Compared to classification, semantic segmentation tasks require much more intensive labeling costs. Thus, these tasks greatly benefit from data-efficient training methods. However, structured outputs in segmentation render particular difficulties (e.g., designing pseudo-labeling and augmentation) to apply existing SSL strategies. To address this problem, we present a simple and novel re-design of pseudo-labeling to generate well-calibrated structured pseudo labels for training with unlabeled or weakly-labeled data. Our proposed pseudo-labeling strategy is network structure agnostic to apply in a one-stage consistency training framework. We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes. Extensive experiments have validated that pseudo labels generated from wisely fusing diverse sources and strong data augmentation are crucial to consistency training for segmentation. The source code is available at this https URL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/217dd8145b5fd7ae7eaa921fe1e98b03039904ad

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

ST++: Make Self-trainingWork Better for Semi-supervised Semantic Segmentation (2021, methodology): “Therefore, also inspired by the success in SSL, subsequent methods [1, 12, 18, 24, 25, 31, 41, 55, 59, 61, 62, 66] manage to tackle this task with simpler mechanisms, such as enforcing similar predictions under multiple perturbed embeddings [41], under two different contextual crops [31], and betwee”
HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation (2022, neutral): “Therefore, several methods aim to reduce the annotation burden such as weakly-supervised learning [14, 61, 93], semi-supervised learning [17, 27, 35, 53], and unsupervised domain adaption (UDA) [26, 28, 59, 67, 92].”
Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision (2021, methodology): “PseudoSeg [44], concurrent to our work, also explores pseudo segmentation for semi-supervised segmentation.”
Continual Semantic Segmentation Leveraging Image-level Labels and Rehearsal (2022, methodology): “Our first intuition is to design a dedicated branch that is specialized in the production of pseudo-labels from weakly-labelled data and to learn from a consistency loss [Zou et al. , 2021].”
Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation (2022, methodology): “As demonstrated in our main paper, our UniMatch has achieved remarkable improvement against the only two methods [81, 78] that provide results on the challenging COCO dataset.”
Novel Class Discovery in Semantic Segmentation (2021, methodology): “SSL [4, 43] utilizes a proportion of annotated data, aiming to achieve comparable performance with fully-supervised methods.”
Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels (2022, methodology): “To mitigate the problem of confirmation bias [2], where the model may suffer from incorrect pseudo-labels, existing approaches propose to filter the predictions with their confidence scores [43, 44, 51, 52].”
Extract Free Dense Labels from CLIP (2021, methodology): “Self-training has firstly become popular in the semisupervised classification task [33,29,35,43] and is also recently applied in the semi-supervised/zero-shot semantic segmentation settings [27,37,4,28,40,36,56,8].”
Bootstrapping Semantic Segmentation with Regional Contrast (2021, methodology): “0] , following PseudoSeg’s training setup (Zou et al., 2021).”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, methodology): “…UL for image classiﬁcation, and adapt techniques such as consistency regularization [31], [178], [179], [180], self-training [32], [168], [174], [181], [182], [183], [184], GAN frameworks [185], [186] in SSL, or contrastive learning [187], [188], [189], [190] in UL to learn from unlabeled images.”