Constructing informative priors using transfer learning

By Rajat Raina, Andrew Y. Ng, Daphne Koller. 2006

331 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Recasting Gradient-Based Meta-Learning as Hierarchical Bayes (2018, methodology): “ Lawrence& Platt, 2004; Yu et al., 2005; Gao et al., 2008; Daumé III, 2009; Wan et al., 2012). A related subﬁeld is that of transfer learning, which has used hierarchical Bayes extensively (e.g., Raina et al., 2006). A variety of inference methods have been used in Bayesian models, including exa”
A Convex Formulation for Learning Task Relationships in Multi-Task Learning (2010, background): “Most of the above methods focus on symmetric multi-task learning, but there also exist some previous works that study asymmetric multi-task learning [23, 19, 10].”
Feature-based transfer learning with real-world applications (2010, background): “Web applications of transfer learning include text classification across domains [151, 47, 48, 72, 195, 207, 36, 204, 213, 203], advertising [40, 39], learn to rank across domains [10, 192, 35, 68] and multi-domain collaborative filtering [103, 102, 29, 223, 143].”
Importance of Semantic Representation: Dataless Classification (2008, unknown): “( Raina, Ng, & Koller 2006 ) used the 20 Newsgroup dataset to construct ten binary classification problems, shown in Table 3. We used these problems for our experiments.”
Online Methods for Multi-Domain Learning and Adaptation (2008, background): “Others have considered transfer learning, in which an existing domain is used to improve learning in a new domain, such as constructing priors (Raina et al., 2006; Marx et al., 2008) and learning parameter functions for text classiﬁcation from related data (Do and Ng, 2006).”
Transferring Naive Bayes Classifiers for Text Classification (2007, background): “Raina, Ng, & Koller (2006) constructed informative priors by discovering highly correlated words, and then used these priors to enhance the classiﬁcation with limited labeled data.”
Multi-domain learning by confidence-weighted parameter combination (2010, methodology): “Raina et al. (2006) obtain these statistics from data from related problems.”
A Hierarchical Bayesian Model for Unsupervised Induction of Script Knowledge (2014, background): “Raina et al. (2006) encode word correlations in a variance-covariance matrix of a multivariate normal distribution (MVN), and sample prior parameter vectors from it, thus introducing dependencies among the parameters.”
Translated Learning: Transfer Learning across Different Feature Spaces (2008, background): “semi-supervis ed learning [13] and transfer learning [3, 11, 10, 9, 4].”
Towards semantic knowledge propagation from text corpus to web images (2011, background): “HTL (Heterogeneous Transfer Learning)[18]: This algorithm is the best fit to our scenario with heterogenous spaces compared to other transfer learning algorithms such as [13][12] on a homogeneous space.”
Self-taught clustering (2008, background): “Raina et al. (2006) proposed to learn logistic regression classifiers by incorporating labeled data from irrelevant categories through constructing informative prior from the irrelevant labeled data.”
Transfer defect learning (2013, background): “In terms of applications, transfer learning has been widely studied to address cross-domain problems in text classification [15], [53], [54], natural language processing [25], [50], WiFi-based indoor localization [55], and computer vision [56].”
Risk Bounds for Transferring Representations With and Without Fine-Tuning (2017, methodology): “The technique is novel to our knowledge, although other approaches to transferring regularization between tasks exist (Evgeniou & Pontil, 2004; Raina et al., 2006; Argyriou et al., 2008; Ghifary et al., 2014).”
Online Learning of Multiple Tasks and Their Relationships (2011, background): “20newsgroups, constructed as in [Raina et al., 2006] contains a total of 11269 training and 7505 test examples for 10 tasks.”
Generalization Bounds for Transfer Learning under Model Shift (2015, background): “Do & Ng (2005) and Raina et al. (2006) assume that models for related tasks share same parameters or prior distributions of hyperparameters.”
Differentially Private Hypothesis Transfer Learning (2018, background): “The first data set is the text classification data set 20NewsGroup (20NG)3 [21], a popular benchmark data set for transfer learning and domain adaptation [33,7,29,23].”