Skill Discovery in Continuous Reinforcement Learning Domains using Skill Chaining

By George Dimitri Konidaris, Andrew Barto. 2009

381 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Learning Parameterized Skills (2012, background): “mination conditions, for instance, might be determined by identifying possibly useful subgoal states [47, 14, 16, 38] that an agent may want to reach.”
The Option-Critic Architecture (2016, background): “In (Konidaris and Barto 2009), an option can only be used and updated after a gestation period of 10 episodes.”
Deep Successor Reinforcement Learning (2016, background): “There has also been a lot of work on option discovery in the tabular setting [14, 18–20 , 34 Our model is also related to the literature on value function approximation using deep neural networks.”
Autonomous Skill Acquisition on a Mobile Manipulator (2011, background): “This use of skill-specific abstractions is a key advantage of hierarchical RL because it allows problems that are highdimensional when considered monolithically to be adaptively broken into subtasks that may themselves be lowdimensional (Konidaris and Barto 2009a).”
A Multitask Representation Using Reusable Local Policy Templates (2012, background): “Pre-dictable results with performance bounds enable capabilities to be used as controllers which direct the agent towards desired regions of state space in the spirit of funnels (Burridge, Rizzi, and Koditschek 1999) or skill chaining (Konidaris and Barto 2009).”
From Skills to Symbols: Learning Symbolic Representations for Abstract High-Level Planning (2018, methodology): “We use 33 continuous features computed from the state variables, describing the x and y distance between each of the agent’s effectors and each object instead of their absolute positions, plus the music, light, and monkey scream, as is standard for this domain (Konidaris & Barto, 2009a).”
Using dimensionality reduction to exploit constraints in reinforcement learning (2010, background): “Despite recent advances [4], problems remain, in particula r with large state spaces.”
Eigenoption Discovery through the Deep Successor Representation (2017, background): “Option discovery has received ample attention over many years, with varied solutions being proposed ( e.g. , Bacon et al., 2017; S¸imsek & Barto, 2004; Daniel et al., 2016; Florensa et al., 2017; Konidaris & Barto, 2009; Mankowitz et al., 2016; McGovern & Barto, 2001).”
Learning Robust Options (2018, methodology): “A Reinforcement Learning option (Sutton, Precup, and Singh 1999; Konidaris and Barto 2009) consists of a three-tuple ζ i = (cid:2) I, ξ χ i , β ( x ) (cid”
Constructing Skill Trees for Reinforcement Learning Agents from Demonstration Trajectories (2010, methodology): “To the best of our knowledge (see Section 6), skill chaining [3] is the only such method that does not make any assumptions about the domain structure.”
Source Task Creation for Curriculum Learning (2016, methodology): “This idea is similar to skill chaining [8], except that instead of learning options linking target regions to initiation sets, we link directly on tasks.”
Multi-Level Discovery of Deep Options (2017, background): “…transitional states [31, 32, 33, 34, 6], inference from demonstrations [8, 35, 5, 12], iteratively expanding the set of solvable initial states [36, 11], policy gradient [37], trading off value with informational constraints [38, 39, 40, 41], active learning [7], or recently value-function…”
GRAIL: A Goal-Discovering Robotic Architecture for Intrinsically-Motivated Learning (2016, background): “Some efforts have been made toward this direction in the field of hierarchical reinforcement learning [13] but some of these works [2], [41], [48] focus on searching for subgoals on the basis of some sort of externally given tasks.”
Toward Good Abstractions for Lifelong Learning (2017, background): “Action abstraction has appeared under a diversity of names, including skills [17], macrooperators [19, 22], and temporal abstraction.”
Curriculum Learning for Motor Skills (2012, background): “Stout and Barto [14] explore the use of a competence-based intrinsic motivation as a developmental model for skill acquisition.”