Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation

By Ruijia Xu, Guanbin Li, Jihan Yang, Liang Lin. IEEE International Conference on Computer Vision, 2018

574 citations recorded by the source corpus.

Domain adaptation enables the learner to safely generalize into novel environments by mitigating domain shifts across distributions. Previous works may not effectively uncover the underlying reasons that would lead to the drastic model degradation on the target task. In this paper, we empirically reveal that the erratic discrimination of the target domain mainly stems from its much smaller feature norms with respect to that of the source domain. To this end, we propose a novel parameter-free Adaptive Feature Norm approach. We demonstrate that progressively adapting the feature norms of the two domains to a large range of values can result in significant transfer gains, implying that those task-specific features with larger norms are more transferable. Our method successfully unifies the computation of both standard and partial domain adaptation with more robustness against the negative transfer issue. Without bells and whistles but a few lines of code, our method substantially lifts the performance on the target task and exceeds state-of-the-arts by a large margin (11.5% on Office-Home and 17.1% on VisDA2017). We hope our simple yet effective approach will shed some light on the future research of transfer learning. Code is available at https://github.com/jihanyang/AFN.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/3b1ec25df36c299659a5cc318907ae658a001da5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Stochastic Classifiers for Unsupervised Domain Adaptation (2020, unknown): “2 SAFN [57] 93.”
Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation (2021, background): “AFN [52] shows that the erratic discrimination of target features stems from much smaller norms than those found in the source features.”
Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering (2020, methodology): “To enforce the cluster assumption, conditional entropy minimization [20, 32] is widely used in the UDA community [11, 44, 45, 50, 51, 60, 64, 68].”
A Balanced and Uncertainty-aware Approach for Partial Domain Adaptation (2020, methodology): “sider integrating them with DANN and CDAN [26] respectively, and compare our methods with state-of-the-art UDA approaches including [44,46] on the most-favored Office-Home dataset.”
Generalized Source-free Domain Adaptation (2021, unknown): “AFN [44] shows that the erratic discrimination of target features stems from much smaller norms than those found in source features.”
CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation (2021, methodology): “…2020; He et al. 2021; Khan et al. 2021), it is found that cross-attention in Transformer is good at aligning different distributions, even from different modalities e.g., vision-to-vision (Li et al. 2021e), vision-to-text (Tsai et al. 2019; Hu and Singh 2021) and text-to-speech (Li et al. 2019).”
ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection (2021, methodology): “[45] propose a progressive feature-norm enlarging method to reduce the domain gap.”
Source Data-Absent Unsupervised Domain Adaptation Through Hypothesis Transfer and Labeling Transfer (2020, methodology): “been exploited in many previous UDA methods [41], [82].”
Unsupervised Domain Adaptation in Semantic Segmentation via Orthogonal and Clustered Embeddings (2020, background): “A variety of techniques have been proposed to tackle the UDA task, ranging from completely non-adversarial approaches [67, 16, 1, 68, 12, 43] to enhanced adversarial strategies [14, 63].”
Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation (2020, methodology): “The results of source model only are especially boosted with the help of BN that is also adopted in prior studies (Xu et al., 2019; Wang et al., 2019).”
Universal Domain Adaptation through Self Supervision (2020, methodology): “set, we use Ofﬁce [33], which has three domains (Amazon (A), DSLR (D), Webcam (W)) and 31 classes. The second benchmark dataset OfﬁceHome 5 Table 2: Results on closed domain adaptation including SAFN [43], CDAN [26] and MDD [46]. Universal comparison Method Ofﬁce (31 / 0 / 0) Ofﬁce-Home (65 / 0 / 0)”
Learning to Generate Novel Domains for Domain Generalization (2020, unknown): “One line of research towards alleviating the domain shift problem is unsupervised domain adaptation (UDA), which exploits unlabeled target domain data for model adaptation [12,33,20,40,53,44].”
Cross-Domain Gradient Discrepancy Minimization for Unsupervised Domain Adaptation (2021, methodology): “org/2014/adaptation use data augmentation such as the ten-crop ensemble used in [25, 40] during the evaluation, for fair comparison, the experimental results cited from previous works are also the versions without data augmentation.”
Towards Discriminability and Diversity: Batch Nuclear-Norm Maximization Under Label Insufficient Situations (2020, unknown): “1 SAFN [52] 52.”
Cross-Domain First Person Audio-Visual Action Recognition through Relative Norm Alignment (2021, methodology): “We can divide UDA approaches in discrepancy-based methods, which explicitly minimize a distance metric among source and target distributions [60, 45], e.”
Adversarial Domain Adaptation With Prototype-Based Normalized Output Conditioner (2020, methodology): “Besides, we also compare with the state-of-the-art works including SAFN [46], SWD [49], ADR [47], DWC-MEC [45], BSP [48], CAT [66] and iCAN [67].”
Minimum Class Confusion for Versatile Domain Adaptation (2019, methodology): “AFN [49] enlarges feature norm to enhance feature transferability.”