To transfer or not to transfer

By Michael T. Rosenstein, Zvika Marx, Leslie Pack Kaelbling, Thomas G. Dietterich. 2005

557 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, background): “In this section we investigate the effect of data imbalance across languages on these learning dynamics, particularly through the lens of transfer and interference (Caruana, 1997; Rosenstein et al., 2005).”
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning (2021, background): “In many cases, it is not unreasonable to expect negative transfer (Rosenstein et al., 2005; Vu et al., 2020).”
Strategies for Pre-training Graph Neural Networks (2019, background): “Otherwise, the transfer of knowledge from related pre-training tasks to a new downstream task can harm generalization, which is known as negative transfer (Rosenstein et al., 2005) and signiﬁcantly limits the applicability and reliability of pre-trained models.”
Multi-Source Domain Adaptation: A Causal View (2015, background): “If the data distribution changes arbitrarily across domains, clearly knowledge from the sources may not help in predicting Y on the target domain (Rosenstein et al. 2005).”
Meta-Learning: A Survey (2018, neutral): “, 2005; Evgeniou and Pontil, 2004), parametric Bayesian models (Rosenstein et al., 2005; Raina et al., 2006; Bakker and Heskes, 2003), Bayesian networks (Niculescu-Mizil and Caruana, 2005), clustering (Thrun, 1998) and reinforcement learning (Hengst, 2002; Dietterich et al.”
Learning Problem-agnostic Speech Representations from Multiple Self-supervised Tasks (2019, background): “Moreover, if not learned with a large pool of tasks [1], supervised representations are likely to be biased towards the considered problem, limiting their exportability to other problems and applications [2].”
Learning to select data for transfer learning with Bayesian Optimization (2017, methodology): “Rather than ordering data (in their case, paragraphs), we use Bayesian Optimization for learning to select relevant training instances that are useful for transfer learning in order to prevent negative transfer (Rosenstein et al., 2005).”
Domain adaptation from multiple sources via auxiliary classifiers (2009, background): “The brute-force transfer of knowledge without domain selection may degrade the classi.cation performance in the target domain (Schweikert et al., 2009), which is also known as negative transfer (Rosenstein et al., 2005).”
Phase Transitions in Transfer Learning for High-Dimensional Perceptrons (2021, unknown): “One particular challenge in transfer learning is to avoid the so-called negative transfer [6]–[9], where the transferred source information reduces the generalization performance of the target task.”
Differentially Private Hypothesis Transfer Learning (2018, background): “It is crucial in avoiding negative transfer as brute-force transfer may actually hurt performance if the sources are too dissimilar to the target [35].”
Distant Domain Transfer Learning (2017, background): “In the worst case, it could lead to even worse performance than ‘non-transfer’ algorithms in the target domain, which is the ‘negative transfer’ phenomena (Rosenstein et al. 2005; Pan and Yang 2010).”
Graph Contrastive Learning with Augmentations (2020, background): “During transfer, substantial domain knowledge is required for both pre-training and downstream tasks, otherwise it might lead to negative transfer [9, 44].”
InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization (2019, methodology): “Simply combining the two loss functions using the same encoder may lead to “negative transfer” 1 [44, 50].”
Multi-source domain adaptation and its application to early detection of fatigue (2011, background): “In addition, there is a conditional probability difference with the test data, hence the brute-force use of knowledge without evaluating similarities between the source and the target data degrades the performance of the classifier in the target domain [25] related to negative transfer [23].”
On handling negative transfer and imbalanced distributions in multiple source transfer learning (2014, background): “Negative transfer [12] [14] [13] refers to the phenomenon that, instead of improving performance, transfer learning from other domains degrades the performance on the target domain.”
Learning to Recognize Activities from the Wrong View Point (2008, methodology): “One might: use similar tasks to reasonably estimate the prior [48,30,?]; use hierarchical bayesian models with hyper priors constrained to be similar for similar tasks [5,25,44]; consider a regularized multi task learning framework, when there is a trade-off between large margins and similar decisio”
Reconciling meta-learning and continual learning with online mixtures of tasks (2018, background): “On the other hand, transfer in the presence of dissimilar or outlier tasks worsens generalization performance [44, 12].”