Towards more Reliable Transfer Learning

By Zirui Wang, Jaime G. Carbonell. 2018

12 citations recorded by the source corpus.

Multi-source transfer learning has been proven effective when within-target labeled data is scarce. Previous work focuses primarily on exploiting domain similarities and assumes that source domains are richly or at least comparably labeled. While this strong assumption is never true in practice, this paper relaxes it and addresses challenges related to sources with diverse labeling volume and diverse reliability. The first challenge is combining domain similarity and source reliability by proposing a new transfer learning method that utilizes both source-target similarities and inter-source relationships. The second challenge involves pool-based active learning where the oracle is only available in source domains, resulting in an integrated active transfer learning framework that incorporates distribution matching and uncertainty sampling. Extensive experiments on synthetic and two real-world datasets clearly demonstrate the superiority of our proposed methods over several baselines including state-of-the-art transfer learning methods. Code related to this paper is available at: https://github.com/iedwardwangi/ReliableMSTL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yulia Tsvetkov
2. Orhan Firat
3. Zirui Wang
4. Yuan Cao
5. Graham Neubig
6. Mohit Bansal
7. Dongrui Wu
8. Lingfei Deng

In Computer Vision and Pattern Recognition

1. Zirui Wang
2. Yuan Cao
3. Dongrui Wu
4. Lingfei Deng
5. Wen Zhang
6. Orhan Firat
7. Yulia Tsvetkov
8. Lei Zhang

Citations in the corpus

• Efficient Meta Lifelong-Learning with Limited Memory (2020, background): “In addition, as prior work have shown transferring from diversely related sources can hurt performance in the target (Ge et al., 2014; Wang and Carbonell, 2018), we study if transferring from Enc-Dec Replay MbPA++ Meta-MbPA”
• Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models (2020, background): “As prior work have shown training with unbalanced tasks can be prone to negative interference (Ge et al., 2014; Wang & Carbonell, 2018), we study how to mitigate it in large models trained with highly unbalanced and massive-scale dataset.”
• Overcoming Negative Transfer: A Survey (2020, methodology): “[40] proposed a peer-weighted TL approach for multi-source transfer.”
• Characterizing and Avoiding Negative Transfer (2018, background): “A symmetric study of source data can be found in [34].”
• A Survey on Negative Transfer (2020, methodology): “MMD [42] is a nonparametric divergence measure, and can be computed directly from the feature means in the raw feature space or a mapped reproducing kernel Hilbert space (RKHS).”