Autonomous discovery of temporal abstractions from interaction with an environment

By Elizabeth McGovern, Andrew Barto. 2002

98 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Investigating Contingency Awareness Using Atari 2600 Games (2012, background): “It also seems plausible that contingency awareness could be useful for option discovery (McGovern 2002), for example by restricting option policies to only use contingent regions as state information.”
Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning (2017, background): “This makes it hard to evaluate the agent’s ability to compose pre-learned policies to solve previously unseen sequential tasks in a zero-shot fashion unless we re-train the agent on the new tasks in a transfer learning setting (Singh, 1991; 1992; McGovern and Barto, 2002).”
Recent Advances in Hierarchical Reinforcement Learning (2003, methodology): “McGovern [43] [44] developed a method for automatically identifying potentially useful subgoals by detecting regions that the agent visits frequently on successful trajectories but not on unsuccessful trajectories.”
Symmetry Learning for Function Approximation in Reinforcement Learning (2017, unknown): “[15, 16] have tried to automate this process by learning from agent observations but their approach is suboptimal as they fail to discover all the abstractions [12].”
Intrinsically Motivated Reinforcement Learning (2004, background): “Many researchers have recognized the desirability of automatically creating options, and several approaches have recently been proposed (e.g., [5]).”
Tree based hierarchical reinforcement learning (2002, methodology): “The work of McGovern and Barto (2001) and McGovern (2002) builds directly upon the options of Precup (2000).”
Hierarchically organized behavior and its neural foundations: A reinforcement-learning perspective (2008, background): “…problem, or a series of interrelated problems, it keeps a record of states or subsequences that occur relatively frequently in trajectories that culminate in reward (McGovern, 2002; Pickett & Barto, 2002; Thrun & Scwhartz, 1995; see also Minton, Hayes, & Fain, 1985; Yamada & Tsuji , 1989).”
Using relative novelty to identify useful temporal abstractions in reinforcement learning (2004, methodology): “The main difference between our subgoals and those of McGovern (2002) and McGovern and Barto (2001) is the role of the reward function.”
TTree: Tree-Based State Generalization with Temporally Abstract Actions (2002, methodology): “In contrast to the description of McGovern [16], where macros are used to move the agent through bottlenecks and hence move the agent to another tightly connected component of the state space, we use these exploration abstract actions to make sure we have fully explored the current connected compone”
AUTOMATED DISCOVERY OF OPTIONS IN REINFORCEMENT LEARNING (2003, methodology): “Experience replay from the trajectories is then used to learn the policy of the new option [18][19].”