Maximum Density Divergence for Domain Adaptation

By Jingjing Li, Erpeng Chen, Zhengming Ding, Lei Zhu, Ke Lu, Heng Tao Shen. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020

340 citations recorded by the source corpus.

Unsupervised domain adaptation addresses the problem of transferring knowledge from a well-labeled source domain to an unlabeled target domain where the two domains have distinctive data distributions. Thus, the essence of domain adaptation is to mitigate the distribution divergence between the two domains. The state-of-the-art methods practice this very idea by either conducting adversarial training or minimizing a metric which defines the distribution gaps. In this paper, we propose a new domain adaptation method named adversarial tight match (ATM) which enjoys the benefits of both adversarial training and metric learning. Specifically, at first, we propose a novel distance loss, named maximum density divergence (MDD), to quantify the distribution divergence. MDD minimizes the inter-domain divergence (“match” in ATM) and maximizes the intra-class density (“tight” in ATM). Then, to address the equilibrium challenge issue in adversarial domain adaptation, we consider leveraging the proposed MDD into adversarial domain adaptation framework. At last, we tailor the proposed MDD as a practical learning loss and report our ATM. Both empirical evaluation and theoretical analysis are reported to verify the effectiveness of the proposed method. The experimental results on four benchmarks, both classical and large-scale, show that our method is able to achieve new state-of-the-art performance on most evaluations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/a095f92ff85f2cc635b3ba7279fec691a05c2ad8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhen Fang
2. Gan Sun
3. Peidong Jia
4. Lei Zhu
5. Yang Cong
6. Ke Lu
7. Hongzu Su
8. Jiaming Liu

In Computer Vision and Pattern Recognition

1. Zhen Fang
2. Gan Sun
3. Peidong Jia
4. Lei Zhu
5. Yang Cong
6. Ke Lu
7. Hongzu Su
8. Jiaming Liu

In Radiology, Nuclear Medicine and Imaging

1. Zhen Fang
2. Shanghang Zhang
3. Zhengming Ding
4. Yandong Guo

Citations in the corpus

• Where and How to Transfer: Knowledge Aggregation-Induced Transferability Perception for Unsupervised Domain Adaptation (2021, methodology): “Consequently, unsupervised domain adaptation methods [24], [25], [26], [27], [42] have been used to narrow the domain gap in the semantic segmentation tasks.”
• Cross-Domain Gradient Discrepancy Minimization for Unsupervised Domain Adaptation (2021, background): “In addition, central moment discrepancy (CMD) [42] and maximum density divergence (MDD) [16] are another two criterias to align feature distributions in hidden layers.”
• Memory-Efficient Class-Incremental Learning for Image Classification (2020, neutral): “Besides, MDD [81] pays attention to the loss design (i.e. maximum density divergence) to quantify the distribution divergence for adversarial training.”
• ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation (2023, background): “In a given domain, if the intra-class divergence for each category is smaller, it demonstrates that the model has a better understanding of the current distribution (Li et al., 2020).”