Fast Generalized Distillation for Semi-Supervised Domain Adaptation

By Shuang Ao, Xiang Li, Charles X. F. Ling. 2017

68 citations recorded by the source corpus.

Semi-supervised domain adaptation (SDA) is a typical setting when we face the problem of domain adaptation in real applications. How to effectively utilize the unlabeled data is an important issue in SDA. Previous work requires access to the source data to measure the data distribution mismatch, which is ineffective when the size of the source data is relatively large. In this paper, we propose a new paradigm, called Generalized Distillation Semi-supervised Domain Adaptation (GDSDA). We show that without accessing the source data, GDSDA can effectively utilize the unlabeled data to transfer the knowledge from the source models. Then we propose GDSDA-SVM which uses SVM as the base classifier and can efficiently solve the SDA problem. Experimental results show that GDSDA-SVM can effectively utilize the unlabeled data to transfer the knowledge between different domains under the SDA setting.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Matthew Riemer
2. Zhen Fang
3. Tim Klinger
4. Jie Lu
5. Djallel Bouneffouf
6. Michele M. Franceschini
7. Jaemin Na
8. Chen Gong

In Computer Vision and Pattern Recognition

1. Liangpei Zhang
2. Bo Du
3. Lefei Zhang
4. Qian Zhang
5. Jaemin Na
6. Zhen Fang
7. Chen Gong
8. David Edwin Carlson

Citations in the corpus

• Scalable Recollections for Continual Lifelong Learning (2017, methodology): “Additionally, unlabelled data has been widely used (Riemer et al., 2016; Laine & Aila, 2017; Ao et al., 2017; Kulkarni et al., 2017) for knowledge distillation.”
• Self-Ensembling Attention Networks: Addressing Domain Shift for Semantic Segmentation (2019, methodology): “This technique is also known as domain adaptation, which can be further divided into supervised one (Tzeng et al. 2014), unsupervised one (Adel, Zhao, and Wong 2017), and semi-supervised one (Ao, Li, and Ling 2017).”
• Extracting Relationships by Multi-Domain Matching (2018, background): “[15, 14, 1, 3, 39] focus on image domain transfer learning.”
• FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptation (2020, background): “Basically, semi-supervised domain adaptation has more information about some target labels compared with UDA, and some related works [1, 30, 28, 19, 44] have been proposed leveraging semi-supervised signals.”
• Generative Knowledge Distillation for General Purpose Function Compression (2017, background): “Additionally, unlabelled data has been widely used [26, 16, 1, 15] for knowledge distillation.”
• Clarinet: A One-step Approach Towards Budget-friendly Unsupervised Domain Adaptation (2020, background): “, 2016], semi-supervised DA [Ao et al., 2017; Zhou et al., 2019a] and unsupervised DA (UDA) [Gong et al.”
• Teaching Semi-Supervised Classifier via Generalized Distillation (2018, methodology): “Algorithmically, GD has been employed for domain adaptation when the data in source domain are not accessible [Ao et al. , 2017].”
• A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, unknown): “Other methods [60], [61] extend this approach to a semi-supervised scenario”