Bayesian Multitask Learning with Latent Hierarchies

By Hal Daumé Iii. 2009

133 citations recorded by the source corpus.

We learn multiple hypotheses for related tasks under a latent hierarchical relationship between tasks. We exploit the intuition that for domain adaptation, we wish to share classifier structure, but for multitask learning, we wish to share covariance structure. Our hierarchical model is seen to subsume several previously proposed multitask learning models and performs well on three distinct real-world data sets.

Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Probabilistic & Bayesian Methods

In Meta & Continual Learning

Citations in the corpus

Recasting Gradient-Based Meta-Learning as Hierarchical Bayes (2018, background): “Hierarchical Bayesian models have a long history of use in both transfer learning and domain adaptation ( e.g., Lawrence & Platt, 2004; Yu et al., 2005; Gao et al., 2008; Daumé III, 2009; Wan et al., 2012).”
Probabilistic Model-Agnostic Meta-Learning (2018, background): “Hierarchical Bayesian models are a long-standing approach for few-shot learning that naturally allow for the ability to reason about uncertainty over functions [39, 7, 25, 43, 12, 4, 41].”
Uncertainty in Multitask Transfer Learning (2018, background): “Hierarchical Bayes algorithms for multitask learning has a long history [8, 32, 2].”
Bayesian Model-Agnostic Meta-Learning (2018, background): “There have been many studies in the past that formulate meta-learning and learning-to-learn from a probabilistic modeling perspective (Tenenbaum, 1999; Fe-Fei et al., 2003; Lawrence & Platt, 2004; Daumé III, 2009).”
Learning Task Grouping and Overlap in Multi-task Learning (2012, background): “…lie close to each other in some geometric sense (Evgeniou & Pontil, 2004), or parameters share a common prior (Yu et al., 2005; Lee et al., 2007; Daum´e III, 2009), or they lie in a low dimensional subspace (Argyriou et al., 2008a) or on a mani-Appearing in Proceedings of the 29 th…”
A Convex Formulation for Learning Task Relationships in Multi-Task Learning (2010, background): “Moreover, some Bayesian models have been proposed for multi-task learning [7, 15, 27].”
Learning Multi-Level Task Groups in Multi-Task Learning (2015, background): “All rights reserved. and Xu 2010; Zhang and Schneider 2010; Zhang and Yeung 2010; Zhang 2013; Zhang and Yeung 2014), and some assume the task structure is hierarchical (Daumé III 2009; Jalali et al. 2010; Görnitz et al. 2011; Lozano and Swirszcz 2012; Zweig and Weinshall 2013).”
$\ell_{p}-\ell_{q}$ Penalty for Sparse Linear and Sparse Multiple Kernel Multitask Learning (2011, methodology): “Prior knowledge on tasks is then translated into an appropriate regularization term or into a hierarchical Bayesian model that can be handled by a learning algorithm [15], [ 21 ], [52].”
Online Multi-Task Learning Framework for Ensemble Forecasting (2017, background): “[29] learned hierarchical Bayesian models that share a common structure parameterized by a covariance matrix.”
Discriminative Transfer Learning with Tree-based Priors (2013, unknown): “A large class of models based on hierarchical Bayesian models have been used for transfer learning [20, 4, 1, 3, 5].”
Multi-task Domain Adaptation for Sequence Tagging (2016, unknown): “…mostly follows two approaches: parameter tying (i.e. linking similar features during learning) (Dredze and Cram-mer, 2008; Daum´e III, 2007, 2009; Finkel and Manning, 2009; Kumar et al., 2010; Dredze et al., 2010), and learning cross domain representations (Blitzer et al., 2006, 2007;…”
Bias reformulation for one-shot function induction (2014, unknown): “More recently, multitask learning has been framed as inference in a hierarchical Bayesian model [20, 4].”
Learning Multiple Tasks using Manifold Regularization (2010, background): “Recently, it has been shown that learning multiple tasks together helps learning [8] [19] [9] when the tasks are related, and one is able to use an appropriate notion of task relatedness.”
Non-Linear Domain Adaptation with Boosting (2013, methodology): “This idea was later further generalized to allow for more generic forms [14, 16, 17, 15], as in [14] that investigated the use of a hierarchically combined decision boundary.”
GSpartan: a Geospatio-Temporal Multi-task Learning Framework for Multi-location Prediction (2016, methodology): “The predictor variables for building the local models were obtained from NCEP-reanalysis2 data, which is a coarsescale global environmental data that integrates observations with output from a numerical weather prediction model.”