Hierarchical control and learning for markov decision processes

By Ronald E. Parr. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Safe Policy Improvement with Baseline Bootstrapping (2017, methodology): “Theorem 2 is a generalization of Theorem 8 in Petrik et al. (2016). The resulting bounds may look very similar at first.”
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition (1999, methodology): “Parr (1998b) proves that his HAMQ learning algorithm converges with probability 1 to a hierarchically optimal policy.”
A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning (2010, methodology): “Although the formulation is very nice and would match game AI development processes, the underlying solver based on HAMs flattens the task hierarchy by including the program’s memory and call-stack into a new joint-state space, and solves this new MDP instead.”
Recent Advances in Hierarchical Reinforcement Learning (2003, methodology): “To the best of our knowledge (and as pointed out by Parr [48]), the approach most closely related to this in the control literature is that of Forestier and Varaiya [20], which we discuss briefly in Section 4.2.”
Hierarchical multi-agent reinforcement learning (2001, methodology): “Our hierarchical multi-agent RL framework can be viewed as extending the existing single-agent HRL methods, including hierarchies of abstract machines (HAMs) [28], options [38], and MAXQ [9], especially the MAXQ value function decomposition [9], to the cooperative multi-agent setting.”
Using relative novelty to identify useful temporal abstractions in reinforcement learning (2004, methodology): “These include Hierarchies of Abstract Machines (Parr & Russell, 1998; Parr, 1998), MAXQ value function decomposition (Dietterich, 2000), and the options framework (Sutton”
Reinforcement Learning: A Tutorial Survey and Recent Advances (2009, unknown): “, 1998), the MAXQ framework (Dietterich, 2000), and the hierarchical-abstract-machines (HAMs) framework (Parr, 1998; Parr and Russell, 1998) are some of the recent developments in this area.”
Multi-Level Discovery of Deep Options (2017, background): “The ﬁeld of hierarchical reinforcement learning has a long history [9, 1, 10], and has also been applied in robotics [11, 12, 13] and in the analysis of biological systems [14, 15, 16, 17, 18].”
Concurrent Hierarchical Reinforcement Learning (2005, methodology): “In the discounted case, the theorem follows from the result shown in [Parr, 1998].”
Off-Policy Temporal Difference Learning with Function Approximation (2001, background): “Several researchers have argued for an ambitious extension of reinforcement learning ideas into modular, multi-scale, and hierarchical architectures (Sutton, Precup & Singh, 1999; Parr, 1998; Parr & Russell, 1998; Dietterich, 2000).”
Hierarchical Approaches (2012, background): “In the most general case a HAM can be a program that executes any computable mapping of the agent’s complete sensory-action history (Parr, 1998).”
An algebraic approach to abstraction in reinforcement learning (2004, methodology): “It has been shown that under the same conditions as Q-learning, SMDP Q-learning converges to the optimal action value function (Parr, 1998).”
Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEs (2020, background): “There has been a large body of work on continuous-time reinforcement learning, many based on the RL framework of SMDPs [Bradtke and Duff, 1995, Parr and Russell, 1998].”
Discovering Hierarchy in Reinforcement Learning with HEXQ (2002, unknown): “For example, if a robot nav-igates around four equally sized rooms with intercon-necting doorways (Parr, 1998) the state space can be represented by the two variables, room-identiﬁer and position-in-room.”