Asymmetric Tri-training for Unsupervised Domain Adaptation

By Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada. 2017

615 citations recorded by the source corpus.

It is important to apply models trained on a large number of labeled samples to different domains because collecting many labeled samples in various domains is expensive. To learn discriminative representations for the target domain, we assume that artificially labeling the target samples can result in a good representation. Tri-training leverages three classifiers equally to provide pseudo-labels to unlabeled samples; however, the method does not assume labeling samples generated from a different domain. In this paper, we propose the use of an asymmetric tri-training method for unsupervised domain adaptation, where we assign pseudo-labels to unlabeled samples and train the neural networks as if they are true labels. In our work, we use three networks asymmetrically, and by asymmetric, we mean that two networks are used to label unlabeled target samples, and one network is trained by the pseudo-labeled samples to obtain target-discriminative representations. Our proposed method was shown to achieve a state-of-the-art performance on the benchmark digit recognition datasets for domain adaptation.

Subfields: Artificial Intelligence, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

A Comprehensive Survey on Transfer Learning (2019, methodology): “By the way, there are some other methods of providing pseudo labels such as co-training [71], [72] and tri-training [73], [74].”
Adversarial Cross-Domain Action Recognition with Co-Attention (2019, methodology): “Similar to previous work (Saito, Ushiku, and Harada 2017), we also use target videos that have high label prediction conﬁdence as training data, where the predicted pseudo-labels serve as the supervision.”
Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation (2018, methodology): “[34] utilizes tri-training to obtain target pseudo labels.”
Deep Visual Domain Adaptation: A Survey (2018, background): “In [98], two different networks assign pseudo-labels to unlabeled samples, another network is trained by the samples to obtain target discriminative representations.”
Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization (2021, methodology): “To handle the difference, we combine prototypical networks with pseudo-labeling techniques, which are often used in domain transfer literature [42].”
Adversarial Transfer Learning (2018, methodology): “[109] propose an alternative method for unsupervised domain adaptation using three networks.”
Self-Training: A Survey (2022, positive): “While [100] approached this issue by carefully choosing a pseudo-labeling policy, [67] learn a representation via a tri-training scheme, in which the student is trained on target data pseudo-labeled by agreement of two teachers.”
Clarinet: A One-step Approach Towards Budget-friendly Unsupervised Domain Adaptation (2020, methodology): “Please note that, existing UDA methods cannot handle BFUDA, as they require fully-true-label source data [Liu et al., 2020; Saito et al., 2017] or at least 20% true-label source data (demonstrated in [Liu et al.”
An Overview of Deep Semi-Supervised Learning (2020, methodology): “As for unsupervised domain adaptation, semi-supervised methods, such as consistency regularization [142, 95, 47], co-regularization [91] or proxy labeling [134, 133] demonstrated their eﬀectiveness in domain adaptation.”
Open Set Domain Adaptation by Backpropagation (2018, methodology): “Benchmark datasets are released [3], and many methods for unsupervised domain adaptation and semi-supervised domain adaptation have been proposed [4,5,6,7,8,9,10,11].”
KinGDOM: Knowledge-Guided DOMain Adaptation for Sentiment Analysis (2020, methodology): “…the difference between feature representations by utilizing equivalent representation of probability distributions by moment sequences; Asym (Saito et al., 2017) is the asymmetric tri-training framework that uses three neural networks asymmetrically for domain adaptation; MT-Tri (Ruder and…”
A Fully Convolutional Tri-Branch Network (FCTN) for Domain Adaptation (2017, methodology): “Being inspired by the ATDA method [11], we generate pseudo labels by feeding images in the target domain training set to the FCTN and collect predictions from both labeling branches.”
Angular Visual Hardness (2019, background): “Unsupervised domain adaptation (Ben-David et al., 2010) presents an important transfer learning problem and deep self-training (Lee, 2013) recently emerged as a powerful framework to this problem (Saito et al., 2017a; Shu et al., 2018; Zou et al., 2018; 2019).”
Confidence Regularized Self-Training (2019, background): “More recently, self-training with networks emerged as a promising alternative towards domain adaptation [4, 5, 25, 29, 49, 54, 69].”
Racial Faces in the Wild: Reducing Racial Bias by Information Maximization Adaptation Network (2018, methodology): “Previous UDA methods apply the source classiﬁer to predict pseudo-labels in the target domain, by which the network can be ﬁne-tuned using supervised losses [51, 69, 16, 18, 66].”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, background): “Co-training ((5)) is also introduced for UDA, which imputes training targets from multiple co-trained clas-siﬁers to learn from unlabeled data and match cross-domain distributions [209].”