Where and How to Transfer: Knowledge Aggregation-Induced Transferability Perception for Unsupervised Domain Adaptation

By Jiahua Dong, Yang Cong, Gan Sun, Zhen Fang, Zhengming Ding. 2021

153 citations recorded by the source corpus.

Unsupervised domain adaptation without accessing expensive annotation processes of target data has achieved remarkable successes in semantic segmentation. However, most existing state-of-the-art methods cannot explore whether semantic representations across domains are transferable or not, which may result in the negative transfer brought by irrelevant knowledge. To tackle this challenge, in this paper, we develop a novel <underline>K</underline>nowledge <underline>A</underline>ggregation-induced <underline>T</underline>ransferability <underline>P</underline>erception (KATP) module for unsupervised domain adaptation, which is a pioneering attempt to distinguish transferable or untransferable knowledge across domains. Specifically, the KATP module is designed to quantify which semantic knowledge across domains is transferable, by incorporating the transferability information propagation from constructed global category-wise prototypes. Based on KATP, we design a novel KATP Adaptation Network (KATPAN) to determine where and how to transfer. The KATPAN contains a transferable appearance translation module <inline-formula><tex-math notation="LaTeX">$\mathcal {T}_A(\cdot)$</tex-math><alternatives><mml:math><mml:mrow><mml:msub><mml:mi mathvariant="script">T</mml:mi><mml:mi>A</mml:mi></mml:msub><mml:mrow><mml:mo>(</mml:mo><mml:mo>·</mml:mo><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:math><inline-graphic xlink:href="dong-ieq1-3128560.gif"/></alternatives></inline-formula> and a transferable representation augmentation module <inline-formula><tex-math notation="LaTeX">$\mathcal {T}_R(\cdot)$</tex-math><alternatives><mml:math><mml:mrow><mml:msub><mml:mi mathvariant="script">T</mml:mi><mml:mi>R</mml:mi></mml:msub><mml:mrow><mml:mo>(</mml:mo><mml:mo>·</mml:mo><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:math><inline-graphic xlink:href="dong-ieq2-3128560.gif"/></alternatives></inline-formula>, where both modules construct a virtuous circle of performance promotion. <inline-formula><tex-math notation="LaTeX">$\mathcal {T}_A(\cdot)$</tex-math><alternatives><mml:math><mml:mrow><mml:msub><mml:mi mathvariant="script">T</mml:mi><mml:mi>A</mml:mi></mml:msub><mml:mrow><mml:mo>(</mml:mo><mml:mo>·</mml:mo><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:math><inline-graphic xlink:href="dong-ieq3-3128560.gif"/></alternatives></inline-formula> develops a transferability-aware information bottleneck to highlight where to adapt transferable visual characterizations and modality information; <inline-formula><tex-math notation="LaTeX">$\mathcal {T}_R(\cdot)$</tex-math><alternatives><mml:math><mml:mrow><mml:msub><mml:mi mathvariant="script">T</mml:mi><mml:mi>R</mml:mi></mml:msub><mml:mrow><mml:mo>(</mml:mo><mml:mo>·</mml:mo><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:math><inline-graphic xlink:href="dong-ieq4-3128560.gif"/></alternatives></inline-formula> explores how to augment transferable representations while abandoning untransferable information, and promotes the translation performance of <inline-formula><tex-math notation="LaTeX">$\mathcal {T}_A(\cdot)$</tex-math><alternatives><mml:math><mml:mrow><mml:msub><mml:mi mathvariant="script">T</mml:mi><mml:mi>A</mml:mi></mml:msub><mml:mrow><mml:mo>(</mml:mo><mml:mo>·</mml:mo><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:math><inline-graphic xlink:href="dong-ieq5-3128560.gif"/></alternatives></inline-formula> in return. Comprehensive experiments on several representative benchmark datasets and a medical dataset support the state-of-the-art performance of our model.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wen Li
2. Yahao Liu
3. Jinhong Deng
4. Lixin Duan
5. Tong Chu
6. Bo Xu
7. Zhen Fang
8. Shuang Xu

In Computer Vision and Pattern Recognition

1. Wen Li
2. Yahao Liu
3. Jinhong Deng
4. Lixin Duan
5. Tong Chu
6. Zhen Fang
7. Jiale Tao
8. Xiao Wang

Citations in the corpus

• Federated Class-Incremental Learning (2022, background): “[34] integrates unsupervised domain adaptation [5, 9, 27, 28, 57] into federated learning framework [14, 31].”
• TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation (2023, background): “In recent years, deep neural networks (DNNs) have been widely used as a fundamental technology for the development of artificial intelligence, both in established and emerging fields [68, 67, 32, 31, 10].”
• Denoised Maximum Classifier Discrepancy for Source-Free Unsupervised Domain Adaptation (2022, methodology): “While many methods have been proposed to solve the UDA problem (Ganin and Lempitsky 2015; Long et al. 2018; Saito et al. 2018; Dong et al. 2020; Liu et al. 2021; Deng et al. 2021; Dong et al. 2021), these approaches is needed to access the source domain data during the training process.”
• VLP: A Survey on Vision-language Pre-training (2022, background): “Recently, VDBERT[109] modelled the common implicit vision-language alignment in vision and language by pre-training on large-scale image-text pairs via transfer learning[110, 111].”
• Undoing the Damage of Label Shift for Cross-domain Semantic Segmentation (2022, methodology): “Recently, selfsupervised methods, including self-training [16, 17, 32, 49, 51, 52, 54, 55] and semantic consistency [26, 41, 47], have been shown to significantly improve the cross-domain semantic segmentation performance.”
• Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization (2021, background): “DA is also related to improving the generalization ability of models across domains (Ahmed et al., 2021), and while DA can access the target data, DG has no access to any target sample (Xu et al., 2021; Dong et al., 2021).”
• A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal (2022, unknown): “Compared to them, rule-basedandpath-basedKGRmodelsaremoreexplainablebut computation-consumingandlessexpressive[305].”