Multi-task reinforcement learning: a hierarchical Bayesian approach

By Aaron Wilson, Alan Fern, Soumya Ray, Prasad Tadepalli. 2007

354 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Management Science and Operations Research, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Management Science and Operations Research

In Reinforcement Learning

Citations in the corpus

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning (2016, methodology): “Wilson et al. (2007) use hierarchical Bayesian methods to maintain a posterior over possible models of dynamics, and apply optimistic Thompson sampling according to the posterior.”
An Overview of Multi-task Learning (2018, methodology): “Specifically, in [69], each reinforcement learning task is modeled by a Markov decision process (MDP) and different MDPs in all the tasks are related via a hierarchical Bayesian infinite mixture model.”
A Survey on Multi-Task Learning (2017, background): “Some works [141], [142], [143], [144], [145], [146], [147], [148], [149], [150] adapt the ideas introduced in Section 2 to MRL.”
Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, background): “Earlier research on lifelong reinforcement learning worked on cross-domain lifelong reinforcement learning, where each domain has a sequence of similar tasks (Wilson et al., 2007; Bou Ammar et al., 2015).”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, methodology): “One approach (Wilson et al., 2007; Ammar et al., 2014) is to model the distribution over tasks and use this distribution as a prior when a new task is seen.”
Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation (2018, background): “Multi-task methods often aim to capture multiple tasks within the same policy or value function [2, 3].”
Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability (2017, methodology): “MTRL is most beneﬁcial when target tasks share common features (Wilson et al., 2007), and most challenging when the task ID is not explicitly speciﬁed to agents during execution – the setting addressed in this paper.”
Gradient Surgery for Multi-Task Learning (2020, methodology): “…al., 2014; Long & Wang, 2015; Yang & Hospedales, 2016; Sener & Koltun, 2018; Zamir et al., 2018) and reinforcement-learning (Espeholt et al., 2018; Wilson et al., 2007), as well as many different domains, such as vision (Bilen & Vedaldi, 2016; Misra et al., 2016a; Kokkinos, 2017; Liu et al.,…”
Modular meta-learning (2018, methodology): “Previous approaches to meta-learning for robotics have focused on ﬁnding distributions over [3] or initial values of [4, 5] parameters, based on a set of “training tasks,” that will enable a new “test task” to be learned with many fewer training examples.”
Stochastic Neural Networks for Hierarchical Reinforcement Learning (2016, background): “…studied in the past in sequential settings, where the agent is exposed to a sequence of tasks, and should learn to make use of experience gathered from solving earlier tasks to help solve later tasks (Taylor & Stone, 2009; Wilson et al., 2007; Lazaric & Ghavamzadeh, 2010; Devin et al., 2017).”
Deep reinforcement learning with successor features for navigation across similar environments (2016, background): “Historically, most existing work in this direction relied on simple task models and explicitly known relations between tasks or known dynamics [16], [17], [18].”
Neural Predictive Belief Representations (2018, background): “The capability of encoding uncertainty can also be useful for learning policies that efficiently explore partially observable environments by acting to reduce uncertainty on the agent’s belief (as in Bayes-optimal exploration; Wilson et al., 2007; Kolter & Ng, 2009; Sorg et al., 2010; Asmuth & Littm”
MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale (2021, background): “Prior work indicates that multi-task RL can indeed amortize the cost of single-task learning [20, 56, 60, 80, 30].”
Sparse multi-task reinforcement learning (2014, positive): “In [28] the transition and reward kernels of all the tasks are assumed to be generated from a common distribution and samples from different tasks are used to estimate the generative distribution and, thus, improving the inference on each task.”
Neuroevolutionary reinforcement learning for generalized control of simulated helicopters (2011, background): “Though an agent can in principle always compute a Bayes-optimal strategy for exploration [63, 95], doing so is typically computationally intractable.”
Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey (2020, methodology): “layers of a neural network between tasks. In supervised learning, these tasks are dierent classication or regression problems. Similar ideas have been applied in a reinforcement learning context by Wilson et al. (2007). In reinforcement learning, dierent tasks correspond to dierent MDPs. Lifelong le”
Transfer Learning for Reinforcement Learning Domains: A Survey (2009, background): “Second, Wilson et al. (2007) consider learning in a hierarchical Bayesian RL setting.”