Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation

By Chen-Yu Lee, Tanmay Batra, Mohammad Haris Baig, Daniel Ulbricht. 2019

606 citations recorded by the source corpus.

In this work, we connect two distinct concepts for unsupervised domain adaptation: feature distribution alignment between domains by utilizing the task-specific decision boundary and the Wasserstein metric. Our proposed sliced Wasserstein discrepancy (SWD) is designed to capture the natural notion of dissimilarity between the outputs of task-specific classifiers. It provides a geometrically meaningful guidance to detect target samples that are far from the support of the source and enables efficient distribution alignment in an end-to-end trainable fashion. In the experiments, we validate the effectiveness and genericness of our method on digit and sign recognition, image classification, semantic segmentation, and object detection.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Luc Van Gool
2. Jiashi Feng
3. Tao A. Xiang
4. Yu Qiao
5. Trevor Darrell
6. Ziwei Liu
7. Dacheng Tao
8. Chen Change Loy

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Jiashi Feng
4. Xiaogang Wang
5. Trevor Darrell
6. Ming-Hsuan Yang
7. Yi Yang
8. Yu Qiao

In Radiology, Nuclear Medicine and Imaging

1. Jian Liang
2. Jiashi Feng
3. Dapeng Hu
4. Wen Li
5. Diane Joyce Cook
6. Garrett Wilson
7. Trevor Darrell
8. Dequan Wang

Citations in the corpus

• Learngene: From Open-World to Your Learning Task (2021, background): “Transfer learning: Methods such as (Huang et al. 2006; Sugiyama et al. 2008; Sun et al. 2011; Shen et al. 2018; Lee et al. 2019; Duan et al. 2009; Zhuang et al. 2009; Evgeniou, Micchelli, and Pontil 2005), which focus on transferring the knowledge across domains, aim at improving the performance of ”
• Unsupervised Model Adaptation for Continual Semantic Segmentation (2020, methodology): “SWD approximates WD as a summation of WD between a number of random one-dimensional projections: D(p̂J , pT ) ≈ 1 L L∑ l=1 M∑ i=1 |〈γl, zppl[i]〉 − 〈γl, ψ(φ(x t tl[i] ))〉|2 (5) where γl ∈ Sf−1 is uniformly drawn random sample from the unit f -dimensional ball Sf−1, and pl[i] and tl[i] are the sorted ”
• A Comprehensive Survey on Transfer Learning (2019, methodology): “In addition to Table 1, there are some other measurement criteria adopted in transfer learning, including Wasserstein distance [59], [60] and central moment discrepancy [61].”
• What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation (2020, methodology): “Although existing semantic transfer models [25, 28, 29, 36, 38] attempt to narrow the distribution shift among source and target datasets, semantic representations are not all transferable while forcefully taking advantage of”
• A Survey of Unsupervised Deep Domain Adaptation (2018, methodology): “[132] combine target discriminative methods and domain-invariant feature learning with a sliced Wasserstein metric.”
• Domain Generalization: A Survey (2021, background): “This idea has also been extensively studied in the domain adaptation (DA) literature [14], [15], [89], [207], [208], but with a rigorous theoretical support [3].”
• Instance Adaptive Self-Training for Unsupervised Domain Adaptation (2020, background): “Adversarial training for UDA: A large number of UDA schemes [1,13,16,17] are proposed to reduce the domain gap by building shared embedding space to both the source and target domain.”
• Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (2021, background): “As suggested by the theoretical analysis [3], domain alignment methods focus on reducing the distribution mismatch by optimizing some divergence [30, 35] or adopting adversarial training [20, 40] at either the image level [1, 12, 19, 26, 49, 63, 68], the intermediate feature level [6, 27, 48, 58] or”
• Adversarial Domain Adaptation With Prototype-Based Normalized Output Conditioner (2020, methodology): “For synthetic-to-real semantic segmentation tasks, we compare our methods with domain adversarial training methods including AdaptSeg [4], SIBAN [5], AdvEnt [60], CLAN [6], AdaptPatch [19] and another generic UDA method SWD [49].”
• Cross-Domain Gradient Discrepancy Minimization for Unsupervised Domain Adaptation (2021, methodology): “In particular, both MCD and SWD perform significantly poorer than our method with 6.7% and 3.6% lower mean accuracy.”
• Denoised Maximum Classifier Discrepancy for Source-Free Unsupervised Domain Adaptation (2022, methodology): “In addition, there is also a special kind of adversarial method that depends on the adversarial training between the feature extractor and the classifier (Saito et al. 2018; Li et al. 2021; Lee et al. 2019; Lu et al. 2020).”
• A comprehensive review of digital twin — part 1: modeling and twinning enabling technologies (2022, methodology): “Some of the other commonly applied approaches have been relying on the principle of minimizing a divergence-based criterion between source and target distributions, including such criteria as Maximum Mean Discrepancy (MMD) or Wasserstein Discrepancy (Yan et al.”
• Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation (2020, methodology): “For object recognition, we compare with DANN (Ganin & Lempitsky, 2015), DAN (Long et al., 2015), ADR, CDAN, CAT, SAFN (Xu et al., 2019), BSP (Chen et al., 2019), and TransNorm (Wang et al., 2019) and SWD.”