Empirical Bayes for Learning to Learn

By Tom M. Heskes. 2000

111 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

An Overview of Multi-Task Learning in Deep Neural Networks (2017, methodology): “Much other work on learning task relationships for multi-task learning uses Bayesian methods: [Heskes, 2000] propose a Bayesian neural network for multi-task learning by placing a prior on the model parameters to encourage similar parameters across tasks.”
Meta-Learning Probabilistic Inference for Prediction (2018, methodology): “First, we leverage shared statistical structure between tasks via hierarchical probabilistic models developed for multi-task and transfer learning (Heskes, 2000; Bakker and Heskes, 2003).”
Continual Learning with Adaptive Weights (CLAW) (2019, methodology): “Our model is also related to the multi-task learning approach (Caruana, 1997; Heskes, 2000; Bakker & Heskes, 2003; Adel et al., 2017; Zhao et al., 2019a; Stickland & Murray, 2019; Zhao et al., 2019b).”
Task Clustering and Gating for Bayesian Multitask Learning (2003, methodology): “In many applications, this assumption is a little too simplistic and we have to consider more sophisticated priors.”
Meta-Learning in Neural Networks: A Survey (2020, methodology): “1) and span from classic [107] to recent approaches such as”
Computationally Efficient Convolved Multiple Output Gaussian Processes (2011, background): “Despite the fact that there are several other approaches to multitask learning (see for example Caruana, 1997, Heskes, 2000, Bakker and Heskes, 2003, Xue et al., 2007 and references therein), in this paper, we focus our attention to those that address the problem of constructing the covariance or…”
Semi-Supervised Multi-Task Regression (2009, background): “Although there exist some multi-task learning methods which also place a common prior on the model parameters of different tasks [24, 25, 13], to the best of our knowledge none of them is based on GP.”
Tagommenders: connecting users to items through tags (2009, unknown): “Movie-bayes: Movie-bayes is a bayesian generative model for how users rate movies with a particular tag [12].”
An Improved Multi-task Learning Approach with Applications in Medical Diagnosis (2008, background): “From the hierarchical Bayesian viewpoint [7], multitask learning essentially seeks to learn a good prior over all tasks to capture task dependencies.”
Online Learning of Multiple Tasks and Their Relationships (2011, background): “Multitask Learning [Caruana, 1997, Heskes, 2000] refers to the setting when the learner has access to data from multiple related learning tasks.”
Learning Multiple Related Tasks using Latent Independent Component Analysis (2005, background): “Furthermore, our model enjoys a sparsity property which makes it both parsimonious and robust.”
Multitask Sparsity via Maximum Entropy Discrimination (2011, background): “In addition, assume the prior forp(s) is a product of Bernoulli distributions for each element ofs, p(s) = D ∏ d=1 ρs(d)(1−ρ)1−s(d), whereρ is the a priori probability of keeping the features on.”
TaskNorm: Rethinking Batch Normalization for Meta-Learning (2020, background): “A general and useful view of meta-learning is through the perspective of hierarchical probabilistic modelling (Heskes, 2000; Bakker & Heskes, 2003; Grant et al., 2018; Gordon et al., 2019).”
Simultaneously Leveraging Output and Task Structures for Multiple-Output Regression (2012, result): “This is in contrast with existing methods that usually take into account only one of these structures.”
Regularized multi--task learning (2004, methodology): “We compared our method with Hierarchical Bayes (HB) [1, 2] which is considered to be a state of the art method for preference modeling of a population of individual consumers.”
Flexible latent variable models for multi-task learning (2008, background): “On the other hand, from the general Bayesian perspective (Baxter 2000; Heskes 2000) we could treat the problem of learning multiple tasks as learning with a shared Bayesian prior over the task space.”
Learning Multiple Tasks with Kernel Methods (2005, methodology): “Past empirical work has shown that, when there are multiple related learning tasks it is beneficial to learn them simultaneously instead of independently as typically done in practice (Bakker and Heskes, 2003; Caruana, 1997; Heskes, 2000; Thrun and Pratt, 1997).”