PolicyBlocks: An Algorithm for Creating Useful Macro-Actions in Reinforcement Learning

By Marc Pickett, Andrew Barto. 2002

125 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Accelerating Reinforcement Learning with Learned Skill Priors (2020, positive): “The idea of transferring skills between tasks dates back at least to the SKILLS [35] and PolicyBlocks [25] algorithms.”
Using relative novelty to identify useful temporal abstractions in reinforcement learning (2004, background): “Thrun and Schwartz (1995), Pickett and Barto (2002) generate temporal abstractions by finding commonly occurring subpolicies in solutions to a set of tasks.”
An algebraic approach to abstraction in reinforcement learning (2004, background): “Autonomously discovering hierarchical structure is an active topic of research recently (Digney, 1998; McGovern and Barto, 2001; Hengst, 2002; Pickett and Barto, 2002).”
Portable Option Discovery for Automated Learning Transfer in Object-Oriented Markov Decision Processes (2015, methodology): “PolicyBlocks The PolicyBlocks option discovery method [Pickett and Barto, 2002] finds options from a set of previAlgorithm 1 PolicyBlocks Power Merge”
Curriculum Learning for Motor Skills (2012, background): “It is related to reward shaping [12, 8], macro actions [9, 10], hierarchical reinforcement learning, and continuation methods for walking skills [15].”
Hierarchically organized behavior and its neural foundations: A reinforcement-learning perspective (2008, background): “Here, as the agent explores a particular problem, or a series of interrelated problems, it keeps a record of states or subsequences that occur relatively frequently in trajectories that culminate in reward (McGovern, 2002; Pickett & Barto, 2002; Thrun & Scwhartz, 1995; see also Minton, Hayes, & Fa”
A Multitask Representation Using Reusable Local Policy Templates (2012, background): “The question of learning options remains in general an open question, but has been approached from many different angles (Şimşek, Wolfe, and Barto 2005; Pickett and Barto 2002).”
Skill Discovery in Continuous Reinforcement Learning Domains using Skill Chaining (2009, methodology): “Finally, some methods extract options by exploiting commonalities in collections of policies over a single state space [14, 15, 16, 17].”
Learning Purposeful Behaviour in the Absence of Rewards (2016, methodology): “Some authors have also tackled the problem of option dis-Learning covery by trying to identify common subpolicies (Thrun & Schwartz, 1994; Pickett & Barto, 2002), while others proposed methods based on the frequency of the change of state variables (Hengst, 2002).”
Divide and Conquer: Hierarchical Reinforcement Learning and Task Decomposition in Humans (2013, background): “Proposals based on this idea can be found in the work of McGovern and Barto (2001); Picket and Barto (2002); Thrun and Schwartz (1995); Yamada and Tsuji (1989).”
Building Portable Options: Skill Transfer in Reinforcement Learning (2007, background): “…from that taken in prior reinforcement learning research on finding useful macro-actions across sequences of tasks [Bernstein, 1999; Perkins & Precup, 1999; Pickett & Barto, 2002; Thrun & Schwartz, 1995], where the tasks must be in the same state space but may have different reward functions.”
PAC-inspired Option Discovery in Lifelong Reinforcement Learning (2014, methodology): “In our third baseline, we construct a set of options using the PolicyBlocks algorithm (Pickett & Barto, 2002).”