ST++: Make Self-trainingWork Better for Semi-supervised Semantic Segmentation

By Lihe Yang, Wei Zhuo, Lei Qi, Yinghuan Shi, Yang Gao. Computer Vision and Pattern Recognition, 2021

472 citations recorded by the source corpus.

Self-training via pseudo labeling is a conventional, simple, and popular pipeline to leverage unlabeled data. In this work, we first construct a strong baseline of self-training (namely ST) for semi-supervised semantic segmentation via injecting strong data augmentations (SDA) on unlabeled images to alleviate overfitting noisy labels as well as decouple similar predictions between the teacher and student. With this simple mechanism, our ST outperforms all existing methods without any bells and whistles, e.g., iterative retraining. Inspired by the impressive results, we thoroughly investigate the SDA and provide some empirical analysis. Nevertheless, incorrect pseudo labels are still prone to accumulate and degrade the performance. To this end, we further propose an advanced self-training framework (namely ST++), that performs selective re-training via prioritizing reliable unlabeled images based on holistic prediction-level stability. Concretely, several model checkpoints are saved in the first stage supervised training, and the discrepancy of their predictions on the unlabeled image serves as a measurement for reliability. Our image-level selection offers holistic contextual information for learning. We demonstrate that it is more suitable for segmentation than common pixel-wise selection. As a result, ST+ further boosts the performance of our ST. Code is available at https://github.com/LiheYoung/ST-PlusPlus.

Subfields: Artificial Intelligence, Civil and Structural Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/39164d367bd8dc268ba3bf08c4605aacafa1f88c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jiashi Feng
2. Bingyi Kang
3. Hengshuang Zhao
4. Rui Zhao
5. Wayne Zhang
6. Lei Qi
7. Xinyi Le
8. Lihe Yang

In Civil and Structural Engineering

1. Jiashi Feng
2. Hengshuang Zhao

In Computer Vision and Pattern Recognition

1. Jiashi Feng
2. Luc Van Gool
3. Xiaogang Wang
4. Yi Yang
5. Ming-Hsuan Yang
6. Chen Change Loy
7. Dacheng Tao
8. Trevor Darrell

Citations in the corpus

• Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels (2022, methodology): “To mitigate the problem of confirmation bias [2], where the model may suffer from incorrect pseudo-labels, existing approaches propose to filter the predictions with their confidence scores [43, 44, 51, 52].”
• Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data (2024, methodology): “Following prior works [73], instead of fine-tuning S from T , we re-initialize S for better performance.”
• Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation (2022, methodology): “Following closely the research line of semi-supervised learning (SSL), advanced methods in semisupervised semantic segmentation have evolved from GANs-based adversarial training paradigm [21, 56, 47] into the widely adopted consistency regularization framework [19, 49, 81, 12, 28, 29, 62] and reborn”