Planning by Incremental Dynamic Programming

By Richard S. Sutton. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Benchmarking Model-Based Reinforcement Learning (2019, methodology): “In the Dyna algorithm [45, 46, 47], training iterates between two steps.”
Reinforcement Learning: A Survey (1996, background): “2 Dyna Sutton's Dyna architecture [116, 117] exploits a middle ground, yielding strategies that are both more e ective than model-free learning and more computationally e cient than the certainty-equivalence approach.”
Learning Optimal Dialogue Strategies: A Case Study of a Spoken Dialogue Agent for Email (1998, background): “Several reinforcement learning algorithms based on dynamic programming specify a way to calculate ) in terms of the utility of a successor state (Bellman, 1957; Watkins, 1989; Sutton, 1991; Barto et al., 1995).”
Reinforcement Learning: An Introduction (1998, background): “…Remarks 8.1 The overall view of planning and learning presented here has developed gradually over a number of years, in part by the authors (Sutton, 1990, 1991a, 1991b; Barto, Bradtke, and Singh, 1991, 1995; Sutton and Pinette, 1985; Sutton and Barto, 1981b); it has been strongly…”
Nonparametric Model-Based Reinforcement Learning (1997, background): “Model-based approaches learn an explicit model of the system si-multaneously with a value function and policy (Sutton, 1990, 1991a,b; Barto et al.,1995; Kaelbling et al., 1996).”
Model-Augmented Actor-Critic: Backpropagating through Paths (2020, background): “On-policy methods tend to be more stable, but at the cost of sample efficiency (Sutton, 1991; Mnih et al., 2016).”
Inter-module credit assignment in modular reinforcement learning (2003, methodology): “Model-based reinforcement learning is an efficient reinforcement learning algorithm using a state prediction model and a reward model to update the value function (Doya 2000) and select action by longer time step planning(R.S.Sutton 1991).”
Efficient Learning and Planning Within the Dyna Framework (1993, methodology): “All tasks studied involved discrete 2-dimensional maze navigation tasks of the kind studied by Sutton (1990; 1991) .”
Algorithms for Sequential Decision Making (1996, methodology): “Other algorithms use a larger t set to speed up learning: DYNA [156] supplements twith a randomly generated set of states while prioritized sweeping [111] and Queue-DYNA [119] use heuristics to select elements for t that will result in the fastest possibleconvergence of the value function.”
Analysis of Some Incremental Variants of Policy Iteration: First Steps Toward Understanding Actor-Cr (1993, methodology): “In fact, the closestlearning algorithm in the literature to which the results to be obtained here potentially apply isthe Dyna-PI algorithm of Sutton (1990; 1991).”