Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation

By Pan Zhang, Bo Zhang, Ting Zhang, Dong Chen, Yong Wang, Fang Wen. Computer Vision and Pattern Recognition, 2021

592 citations recorded by the source corpus.

Self-training is a competitive approach in domain adaptive segmentation, which trains the network with the pseudo labels on the target domain. However inevitably, the pseudo labels are noisy and the target features are dispersed due to the discrepancy between source and target domains. In this paper, we rely on representative prototypes, the feature centroids of classes, to address the two issues for unsupervised domain adaptation. In particular, we take one step further and exploit the feature distances from prototypes that provide richer information than mere prototypes. Specifically, we use it to estimate the likelihood of pseudo labels to facilitate online correction in the course of training. Meanwhile, we align the prototypical assignments based on relative feature distances for two different views of the same target, producing a more compact target feature space. Moreover, we find that distilling the already learned knowledge to a self-supervised pretrained model further boosts the performance. Our method shows tremendous performance advantage over state-of-the-art methods. The code is available at https://github.com/microsoft/ProDA.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/48764f8f64986cf690ce0f1040b2e6bbf06fd23c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wen Li
2. Luc Van Gool
3. Dengxin Dai
4. Lukas Hoyer
5. Yahao Liu
6. Jinhong Deng
7. Lixin Duan
8. Tong Chu

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dengxin Dai
3. Wen Li
4. Lukas Hoyer
5. Lixin Duan
6. Trevor Darrell
7. Jinhong Deng
8. Ming-Hsuan Yang

Citations in the corpus

• MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation (2022, unknown): “Image ProDA [31] DAFormer [9] HRDA [10] MIC (HRDA) Ground Truth”
• DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation (2021, background): “Image DeepLabV2 Src-Only SegFormer Src-Only ProDA [27] DAFormer (Ours) Ground Truth”
• Undoing the Damage of Label Shift for Cross-domain Semantic Segmentation (2022, methodology): “Combining with self-training Since the self-training methods [32, 44, 49] generally use domain alignment methods as their warm-up model to produce pseudo labels, our model can be easily plugged into them.”
• HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation (2022, unknown): “Possibly, the performance for the mentioned stuff classes could be improved for HRDA by integrating the depth-clues as done in CorDA [71] or pseudo-label prototypes as used in ProDA [87].”