Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density

By Amy McGovern, Andrew Barto. 2001

591 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Learning Parameterized Skills (2012, background): “mination conditions, for instance, might be determined by identifying possibly useful subgoal states [47, 14, 16, 38] that an agent may want to reach.”
Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discovery (2019, background): “Progress in hierarchical learning benefits from a complementary line of work on automatic subgoal discovery [23].”
The Option-Critic Architecture (2016, methodology): “Discovering temporal abstractions autonomously has been the subject of extensive research efforts in the last 15 years (McGovern and Barto 2001; Stolle and Precup 2002; Menache, Mannor, and Shimkin 2002; Şimşek and Barto 2009; Silver and Ciosek 2012), but approaches that can be used naturally with c”
Variational Intrinsic Control (2016, background): “…approach to option learning where the goal is to ﬁnd a small number of options that are useful for a particular task (Sutton et al., 1999; McGovern & Barto, 2001; Stolle & Precup, 2002; Silver & Ciosek, 2012; Kulkarni et al., 2016; Mankowitz et al., 2014; Vezhnevets et al., 2016; Bacon…”
Skill Induction and Planning with Latent Language (2021, background): “Hierarchical policies Hierarchical policy learning and temporal abstraction have been major areas of focus since the earliest research on reinforcement learning and imitation learning (McGovern and Barto, 2001; Konidaris et al., 2012; Daniel et al., 2012).”
Neural Modular Control for Embodied Question Answering (2018, background): “Relative to other work on automatic subgoal discovery in hierarchical reinforcement learning [12–14], we show that given knowledge of the problem structure, simple heuristics are quite effective in breaking down long-range planning into sequential subgoals.”
What can I do here? A Theory of Affordances in Reinforcement Learning (2020, background): “However, intents could also be learned from data—a problem which ties closely to subgoal discovery in temporal abstraction (McGovern & Barto, 2001; Şimşek et al., 2005).”
Source Task Creation for Curriculum Learning (2016, background): “For example, McGovern and Barto [14], identify subgoals as states that occur frequently in successful trajectories .”
Skill Characterization Based on Betweenness (2008, methodology): “More importantly, our characterization serves as a useful guide for developing incremental skill-discovery algorithms that do not rely on knowing or representing the interaction graph in its entirety.”
PolicyBlocks: An Algorithm for Creating Useful Macro-Actions in Reinforcement Learning (2002, methodology): “McGovern (2002, McGovern and Barto 2001) used several of the ideas for this heuristic for a method of subgoal discovery based on finding commonalities across multiple paths to problem solutions.”
Divide and Conquer: Hierarchical Reinforcement Learning and Task Decomposition in Humans (2013, background): “Proposals based on this idea can be found in the work of McGovern and Barto (2001); Picket and Barto (2002); Thrun and Schwartz (1995); Yamada and Tsuji (1989).”