Solving a Huge Number of Similar Tasks: A Combination of Multi-Task Learning and a Hierarchical Bayesian Approach

By Tom M. Heskes. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Recasting Gradient-Based Meta-Learning as Hierarchical Bayes (2018, methodology): “This inductive bias has been implemented in various ways: as learned hyperparameters in a hierarchical Bayesian model that regularize task-specific parameters (Heskes, 1998), as a learned metric space in which to group neighbors (Bottou & Vapnik, 1992), as a trained recurrent neural network that all”
A Model of Inductive Bias Learning (2000, background): “An empirical study showing the utility of the hierarchical Bayes approach in a domain containing a large number of related tasks was given in Heskes (1998).”
Task Clustering and Gating for Bayesian Multitask Learning (2003, neutral): “These results are consistent with the more extended simulation studies in (Heskes, 1998, 2000).”
Bayesian Multitask Inverse Reinforcement Learning (2011, background): “A number of inverse reinforcement learning [1, 5, 7, 16, 18, 20, 23] and preference elicitation [4, 6] approaches have been proposed, while multitask learning itself is a well-known problem, for which hierarchical Bayesian approaches are quite natural [13].”
Empirical Bayes for Learning to Learn (2000, methodology): “This is in con-trast with Heskes (1998), where rst the feature ma-trix B was learned in a standard frequentist manner,after which the hyperparameters for the prior were ob-tained in an (empirical) Bayesian fashion using an EMalgorithm.”
Online Multitask Learning (2006, background): “Much previous work on theoretical and applied multitask learning has concerned how to take advantage of cases in which a number of learning problems are related [3, 4 , 5, 6, 7, 8]; in contrast, we do not assume that the tasks are related and instead we consider how to take account of common conseq”
Machine learning for targeted display advertising: transfer learning in action (2013, unknown): “Alternative labels, candidates for Y S , include: 1) clicking on an ad (still requires showing impressions), 2) any purchase (not just ﬁrst time) after an ad, 3) any purchase with or without an ad, and 4) any other brand action with or without an ad.”
Online Learning of Multiple Tasks with a Shared Loss (2007, background): “…work on theoretical and applied multitask learning has focused on how to take advantage of similarities between the various tasks (Caruana, 1997, Heskes, 1998, Evgeniou et al., 2005, Baxter, 2000, Ben-David and Schuller, 2003, Tsochantaridis et al., 2004); in contrast, we do not assume that the…”
Online gradient-based mixtures for transfer modulation in meta-learning (2018, methodology): “Closest to our proposed approach is early work on hierarchical Bayesian multi-task learning with neural networks (Heskes, 1998; Bakker and Heskes, 2003; Salakhutdinov et al., 2013; Srivastava and Salakhutdinov, 2013).”
On Better Exploring and Exploiting Task Relationships in Multitask Learning: Joint Model and Feature Learning (2018, methodology): “For example, some works extended Bayesian method into multi-task learning methods with the assumption that the models of tasks are indeed related [23, 24].”
Semi-Supervised Multi-Task Regression (2009, background): “Although there exist some multi-task learning methods which also place a common prior on the model parameters of different tasks [24, 25, 13], to the best of our knowledge none of them is based on GP.”
A survey of inverse reinforcement learning techniques (2012, methodology): “Specifically, the algorithm derives preferences from a series of tasks demonstrations using a hierarchical (Heskes, 1998) and empirical Bayesian (Robbins, 1992) approaches that assign multitask priors on reward functions and policies.”