Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation

By Haoran Wang, Tong Shen, Wei Zhang, Ling-Yu Duan, Tao Mei. 2020

340 citations recorded by the source corpus.

Despite great progress in supervised semantic segmentation,a large performance drop is usually observed when deploying the model in the wild. Domain adaptation methods tackle the issue by aligning the source domain and the target domain. However, most existing methods attempt to perform the alignment from a holistic view, ignoring the underlying class-level data structure in the target domain. To fully exploit the supervision in the source domain, we propose a fine-grained adversarial learning strategy for class-level feature alignment while preserving the internal structure of semantics across domains. We adopt a fine-grained domain discriminator that not only plays as a domain distinguisher, but also differentiates domains at class level. The traditional binary domain labels are also generalized to domain encodings as the supervision signal to guide the fine-grained feature alignment. An analysis with Class Center Distance (CCD) validates that our fine-grained adversarial strategy achieves better class-level alignment compared to other state-of-the-art methods. Our method is easy to implement and its effectiveness is evaluated on three classical domain adaptation tasks, i.e., GTA5 to Cityscapes, SYNTHIA to Cityscapes and Cityscapes to Cross-City. Large performance gains show that our method outperforms other global feature alignment based and class-wise alignment based counterparts. The code is publicly available at this https URL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Luc Van Gool
2. Wen Li
3. Dengxin Dai
4. Lukas Hoyer
5. Yahao Liu
6. Jinhong Deng
7. Lixin Duan
8. Lior Wolf

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Ming-Hsuan Yang
3. Trevor Darrell
4. Dacheng Tao
5. Dengxin Dai
6. Wen Li
7. Xiaogang Wang
8. Yi Yang

In Radiology, Nuclear Medicine and Imaging

1. Luc Van Gool
2. Dengxin Dai
3. Lukas Hoyer
4. Haoran Wang
5. Peilin Zhao
6. Jinhong Deng
7. Junzhou Huang
8. Lixin Duan

In Representation & Self-Supervised Learning

1. Wen Li
2. Luc Van Gool
3. Dengxin Dai
4. Zhuo Zheng
5. Peilin Zhao
6. Lior Wolf
7. Jinhong Deng
8. Junzhou Huang

Citations in the corpus

• OpenEarthMap: A Benchmark Dataset for Global High-Resolution Land Cover Mapping (2022, methodology): “[52] Haoran Wang, Tong Shen, Wei Zhang, Lingyu Duan, and Tao Mei.”
• Permuted AdaIN: Reducing the Bias Towards Global Statistics in Image Classification (2020, methodology): “3, our method improves the state of the art when applied in conjunction with FADA [43], achieving a gap of 2 .”
• Undoing the Damage of Label Shift for Cross-domain Semantic Segmentation (2022, methodology): “However, to eliminate the phenomenon of misalignment, many methods [18, 29, 34, 44] have been proposed to conduct the conditional distribution alignment to learn class-level domain-invariant representation.”
• BAPA-Net: Boundary Adaptation and Prototype Alignment for Cross-domain Semantic Segmentation (2021, methodology): “Besides, GAN [50, 38, 37, 13, 6, 54, 25] related works have been used to align domain distribution on feature and output space, respectively.”
• MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation (2022, methodology): “The target loss L is defined according to the UDA strategy such as adversarial training [8, 19, 57, 73, 74, 78] or self-training [30, 53, 72, 89, 90, 97].”
• DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation (2021, methodology): “Several methods also combine adversarial and self-training [38, 42, 78] or train with auxiliary tasks [33, 77, 79].”
• Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (2021, neutral): “These methods could be divided into two categories: 1) domain alignment methods that align the distribution through adversarial training, which include AdaptSeg [54], CyCADA [26], CLAN [36], APODA [67], PatchAlign [56], ADVENT [57], BDL [34] and FADA [60]; 2) self-training approaches, including CBST”
• HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation (2022, positive): “Several works also combine self-training and adversarial training [35, 39, 69, 93], minimize the entropy [7, 67, 68], refine boundaries [41], use a curriculum [13, 14, 89], or learn auxiliary tasks [8, 30, 68, 71].”
• LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation (2021, methodology): “Differing from the binary discriminators, Wang et al. [44] proposed a fine-grained adversarial learning framework for domain adaptive semantic segmentation (FADA), aligning the class-level features.”
• Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation (2021, methodology): “Qualitative comparison of our method against BDL [18], FDA [48], and FADA [41] on the perturbed test data (ε = 0.”