Finding Structure in Reinforcement Learning

By Sebastian Thrun, Anton Schwartz. 1994

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, methodology): “Macro actions (Hauskrecht, Meuleau, Kaelbling, Dean, and Boutilier, 1998) or skills (Thrun and Schwartz, 1995) are approaches for learning that side step the requirement to make”
Accelerating Reinforcement Learning with Learned Skill Priors (2020, positive): “The idea of transferring skills between tasks dates back at least to the SKILLS [35] and PolicyBlocks [25] algorithms.”
A Model of Inductive Bias Learning (2000, background): “…literature have proposed algorithms for both sharing the information in related tasks to improve average generalization performance across those tasks Singh (1992), Ring (1995), or learning bias from a set of tasks to improve performance on future tasks Sutton (1992), Thrun and Schwartz (1995).”
Hierarchical Solution of Markov Decision Processes using Macro-actions (1998, background): “After some ﬁnite number n of iterations, the choice of maximizing action for each s forms an optimal policy (cid:25) and V n approximates its value.”
Recent Advances in Hierarchical Reinforcement Learning (2003, background): “These partial policies are sometimes called temporally-extended actions, options [73], skills [80], behaviors [9] [27], or the more control-theoretic modes [22].”
Hierarchical Approaches (2012, background): “Some of the approaches use to learn abstract actions and structure include searching for common behaviour trajectories or common state region polices (Thrun and Schwartz, 1995; McGovern, 2002).”
Multitask Soft Option Learning (2019, methodology): “The multitask- and transfer-learning setup used in this work is inspired by Thrun and Schwartz (1995) who suggests extracting options by using commonalities between solutions to multiple tasks.”
Composable Planning with Attributes (2018, background): “Hierarchical RL Many researchers have recognized the importance of methods that can divide a MDP into sub-processes (Thrun & Schwartz, 1994; Parr & Russell, 1998; Sutton et al., 1999; Dietterich, 2000).”
Skill Discovery in Continuous Reinforcement Learning Domains using Skill Chaining (2009, methodology): “Finally, some methods extract options by exploiting commonalities in collections of policies over a single state space [14, 15, 16, 17].”
Decision-Theoretic Planning: Structural Assumptions and Computational Leverage (1999, background): “Macros have a long history in classical planning and problem solving [50, 77], but only recently have they been generalized to MDPs [68, 98, 100, 109, 128, 129, 134].”
Transfer in Reinforcement Learning via Shared Features (2012, background): “Second, that representation must facilitategeneralization: in a continuous state space the agent may never encounter the same state twice and must instead generalize from experiences in nearby states when encountering a novel one.”
Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning (1999, methodology): “It is possible to have many such subgoals and learn about them each independently using an off-policy learning method such as Q-learning, as in [17,31,38,66,78].”
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning (1999, methodology): “It is possible to have many such subgoals and learn about them each independently using an off-policy learning method such as Q-learning, as in [17,31,38,66,78].”
A Berkeley View of Systems Challenges for AI (2017, background): “Options have been extensively studied in the context of hierarchical learning [30, 34, 84, 97, 102, 110].”
Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density (2001, background): “For RL systems, several researchers have proposed methods by which policies learned for a set of related tasks are examined for commonalities (Thrun and Schwartz, 1995) or are probabilistically combined to form new policies (Bernstein, 1999).”