Dynamic Programming: Deterministic and Stochastic Models

By Dimitri P. Bertsekas. 1987

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reinforcement Learning: A Survey (1996, background): “Another important result is that the greedy policy is guaranteed to be optimal in some (cid:12)nite number of steps even though the value function may not have converged (Bertsekas, 1987).”
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms (1994, background): “Markovian decision problems can be solved by dynamic programming (Bertsekas, 1987).”
Planning by Incremental Dynamic Programming (1991, background): “DP is a broad class of techniques developed in operations research for solving sequential decision problems such as routing and scheduling (e.g., Bertsekas, 1987).”
Planning algorithms (2006, background): “Most of the ideas were developed in the context of stochastic control theory [95, 567, 570].”
Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules (2016, methodology): “…TD methods (Sutton and Barto, 1998) solve this problem more efficiently using algorithmic ideas inspired by dynamic programming (Bellman, 1957; Bertsekas, 1987): Consistency of state values across different states requires that the expected reward in one state (visited at time t) be equal to…”
Near-Optimal Reinforcement Learning in Polynomial Time (2002, background): “Thus, the reinforcement learning agent faces a fundamental trade-off between exploitation and exploration (Bertsekas, 1987; Kumar & Varaiya, 1986; Thrun, 1992): that is, should the agent exploit its cumulative experience so far, by executing the action that currently seems best, or should it…”
On the undecidability of probabilistic planning and related stochastic optimization problems (2003, methodology): “The reader is referred to books on MDPs and POMDPs, such as Puterman [42], White [49], and Bertsekas [6], for a more comprehensive introduction to the models and motivational examples.”
Efficient approximate planning in continuous space Markovian Decision Problems (2001, unknown): “We assume that the reader is familiar with the basics of the theory of MDPs. Readers who lack the necessary background are referred to the book of Dynkin and Yuskevich [6] or the more recent books [2] and [11].”
Markov Games as a Framework for Multi-Agent Reinforcement Learning (1994, background): “It can be shown that the estimated values for Q and V converge to their true values [Bertsekas, 1987].”
Recent Advances in Hierarchical Reinforcement Learning (2003, unknown): “[4] [5] [55] [58] [72]).”
The Simplex and Policy-Iteration Methods Are Strongly Polynomial for the Markov Decision Problem with a Fixed Discount Rate (2011, methodology): “Bertsekas [3] in 1987 also showed that the value-iteration method converges to the optimal policy in a finite number of iterations.”
Algorithms for Sequential Decision Making (1996, methodology): “Puterman's and Bertsekas' books [126, 16] give in-depth summaries of related work inthis eld.”
Average Reward Reinforcement Learning: Foundations, Algorithms, and Empirical Results (2005, methodology): “Alternatively, periodic MDP's can be transformed using an aperiodicity transformation [5], [28] and then solved using value iteration.”
Adaptive computation and machine learning (1998, methodology): “Several algorithms for low internal regret have been given [Foster and Vohra, 1997, 1998, 1999, Hart and Mas-Colell, 2000, Cesa-Bianchi and Lugosi, 2001, Stoltz and Lugosi, 2003], including a conversion of low external regret to low swap regret by Blum and Mansour [2005]. The Perceptron algorithm wa”