Transfer in variable-reward hierarchical reinforcement learning

By Neville Mehta, Sriraam Natarajan, Prasad Tadepalli, Alan Fern. 2008

102 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Information Systems, Software

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Information Systems

In Software

Citations in the corpus

Disentangling Transfer in Continual Reinforcement Learning (2022, background): “[27, 38] achieve transfer by means of high-level skills and hierarchical RL.”
Successor Features for Transfer in Reinforcement Learning (2016, background): “For example, by covering the relevant subspace of R with balls of appropriate radii centered at wj we can provide performance guarantees for any task w [14].”
Learning by Playing - Solving Sparse Reward Tasks from Scratch (2018, background): “…connected to the broader literature on multi-task (reinforcement) learning (see e.g. Caruana (1997) for a general overview and Lazaric et al. (2008); Mehta et al. (2008) for RL applications) and work on reinforcement learning via options (Dietterich, 1998; Bacon et al., 2017; Daniel et al., 2012).”
Bayesian Reinforcement Learning (2010, background): “While some methods explicitly assume that the shared components are drawn from a common generative model (Wilson et al, 2007; Mehta et al, 2008; Lazaric and Ghavamzadeh, 2010), this assumption is more implicit in others (Taylor et al, 2007; Lazaric et al, 2008).”
Multi-task reinforcement learning: a hierarchical Bayesian approach (2007, background): “Yet other approaches ( Mehta et al., 2005 ) assume that the reward function of the new task is given, and maintain value functions for several dieren t prior tasks and initialize the value function of a new task with one that yields the most reward.”
Bayesian Multi-Task Reinforcement Learning (2010, methodology): “In [10], tasks share the same dynamics and reward features, and only differ in the weights of the reward functi on.”
Sample Complexity of Multi-task Reinforcement Learning (2013, background): “Perhaps most similar to us is Mehta et al. [2008], who consider sample complexity for transfer learning across semi-MDPs with identical ( S, A ) and transition models, and a distribution of reward weight vectors.”
Transfer of samples in batch reinforcement learning (2008, methodology): “In particular, hierarchical solutionsareoften used(S¸im¸sek etal.,2005; Mehta et al., 2005) to augment the action space with policies suitable for the solution of a wide range of tasks sharing the same dynamics, but with di.erent goals.”
Hierarchical Reinforcement Learning: A Survey (2015, background): “Hierarchical Reinforcement Learning (HRL) refers to the notion in which RL problem is decomposed into subproblems (sub-tasks) where solving each of which will be more powerful than solving the entire problem [4], [5], [6] and [27], [36].”
Convex Optimization: Algorithms and Complexity (2015, unknown): “In [Mehta et al., 2008], tasks share the same dynamics and reward features, and only differ in the weights of the reward function.”
Transfer Learning for User Adaptation in Spoken Dialogue Systems (2016, background): “Representation transfer [37] is an active research area: one can find studies on option based transfers [20, 30], which are dependant on hierarchical reinforcement learning [36], on Proto-value functions [29, 15], or on representation transfer for Gaussian Processes Reinforcement Learning [16, 17, 5”
Construction of approximation spaces for reinforcement learning (2013, background): “This class of tasks is also called variable-reward tasks (Mehta et al., 2008) and applies for example when a flying robot needs to execute different maneuvers, but is constraint by the same aerodynamics (Abbeel et al.”
Transfer in Reinforcement Learning: A Framework and a Survey (2012, background): “Let us consider a similar scenario to the real-time strategy (RTS) game introduced in (Mehta et al, 2008).”
Autonomous Agents and Multi-agent Systems Manuscript No. Learning Potential Functions and Their Representations for Multi-task Reinforcement Learning (unknown): “[51] assume fixed task dynamics and reward features, but different reward feature weights in each task.”
Transfer Learning in Deep Reinforcement Learning: A Survey (2020, background): “Accordingly, learning ψ ( s, a ) for any policy π i in M i becomes equivalent to learning a collection of Q-functions: A similar idea of using reward functions as features to represent unseen tasks is also proposed by [129], which assumes the ψ and w as observable quantities from the environment.”
Transfer Learning for Reinforcement Learning Domains: A Survey (2009, background): “However, representation selection is a difficult problem in RLin general and discussions of representation selection (or its applications to transfer efficacy)are beyond the scope of this article.”
Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective (2010, methodology): “Although the variable-reward approach of Mehta et al. [ 21 ] involves multiple reward functions, it is quite different in that the tasks in the distribution differ in their reward functions rather than in other features, and no reward–function search is involved.”
Hierarchical Approaches (2012, methodology): “Mehta et al (2008a) transfer the value function between different SMPDs and find that transfer is especially effective in the hierarchical setting.”