Efficient skill learning using abstraction selection

By George Dimitri Konidaris, Andrew Barto. 2009

109 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Meta & Continual Learning

Citations in the corpus

Learning Parameterized Skills (2012, methodology): “Konidaris and Barto (2007) learn reusable options by representing them in an agent-centered state space but do not address the problem of how to construct skills for solving a family of related tasks.”
Unsupervised state representation learning with robotic priors: a robustness benchmark (2017, methodology): “This is an alternative approach to selecting a state representation from a set ([7], [8]), or creating an autoencoder to compress information into a lower dimensional state ([9], [10]), [11].”
Learning and Transfer of Modulated Locomotor Controllers (2016, background): “The reinforcement learning literature has also explored a wide variety of temporal abstractions that wrap low-level control into options [25] or skills [14].”
Learning state representations with robotic priors (2015, background): “State representation learning is related to the problem of abstraction selection in hierarchical reinforcement learning, which deals with selecting a state representation from a given set [35, 17] or choosing a subset of state dimensions from a given representation [4].”
Constructing Skill Trees for Reinforcement Learning Agents from Demonstration Trajectories (2010, background): “It can be coupled with abstraction selection [4] to select skill-specific abstractions, which can aid in acquiring policies that are high-dimensional when represented monolithically, but can be broken into subpolicies that can be defined over far fewer variables.”
Context-Specific Representation Abstraction for Deep Option Learning (2021, methodology): “Our approach is related to prior work that considers the importance of state abstraction and the decomposition of the learning problem (Konidaris 2019; Konidaris and Barto 2009; Jong and Stone 2005).”
Object focused q-learning for autonomous agents (2013, unknown): “Other works have a set of candidate representations out of which one is chosen for a specific task [12] [14], but this is impractical in high-dimensional domains if there is no previous knowledge about which features are useful together.”
From Skills to Symbols: Learning Symbolic Representations for Abstract High-Level Planning (2018, methodology): “We use 33 continuous features computed from the state variables, describing the x and y distance between each of the agent’s effectors and each object instead of their absolute positions, plus the music, light, and monkey scream, as is standard for this domain (Konidaris & Barto, 2009a).”
Autonomous Skill Acquisition on a Mobile Manipulator (2011, background): “This use of skill-specific abstractions is a key advantage of hierarchical RL because it allows problems that are highdimensional when considered monolithically to be adaptively broken into subtasks that may themselves be lowdimensional (Konidaris and Barto 2009a).”
Abstraction Selection in Model-based Reinforcement Learning (2015, background): “For recent empirical advances on the problem, we refer the reader to Konidaris & Barto (2009), Cobo et al. (2014), and Seijen et al. (2014).”
Composing Meta-Policies for Autonomous Driving Using Hierarchical Deep Reinforcement Learning (2017, methodology): “policies can be constructed with closed loop sub-policies called options [12, 4, 8, 14, 26, 9].”
Skill Discovery in Continuous Reinforcement Learning Domains using Skill Chaining (2009, background): “One such approach is abstraction selection [25, 26], where an agent uses sample trajectories (as obtained during gestation) to select an appropriate abstraction for a new option from a library of candidate abstractions, potentially resulting in a much easier learning problem.”
EFFICIENT ABSTRACTION SELECTION IN REINFORCEMENT LEARNING (2013, methodology): “More recently, methods have emerged that focus on selecting the best abstraction , a subset of state components, from a set of candidate abstractions (Diuk, Li, and Lefﬂer 2009; Konidaris and Barto 2009).”
Exploration from Generalization Mediated by Multiple Controllers (2013, background): “However, work on understanding model-free generalization more broadly, outside the confines of a task such as this, is still in its early days (Konidaris and Barto 2007, 2009).”
Inventing Relational State and Action Abstractions for Effective and Efficient Bilevel Planning (2022, methodology): “We adopt a very general deﬁnition of an abstraction [5]: mappings from X and A to alternative state and action spaces.”
Compositional Planning Using Optimal Option Models (2012, methodology): “In this paper we have focused on planning with table lookup models; however, similar to MAXQ (Dietterich, 2000), HAMs (Andre & Russell, 2002) or skills (Konidaris & Barto, 2009), substantial eﬃciency improvements may be generated when each option model is provided with its own state abstraction.”