Dynamic Programming and Optimal Control, Two Volume Set

By Dimitri P. Bertsekas. 1995

Subfields: Reinforcement Learning, Mathematical Optimization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Mathematical Optimization

Citations in the corpus

Addressing Function Approximation Error in Actor-Critic Methods (2018, background): “The proof of lemma 1 can be found in Singh et al. (2000), building on a proposition from Bertsekas (1995).”
Bridging the Gap Between Value and Policy Based Reinforcement Learning (2017, background): “We then have = max s F τ ( q (1) s ) − F τ ( q (2) s ) ≤ max The existence and uniqueness of V ∗ then follows from the contraction map ﬁxed-point theorem (Bertsekas, 1995).”
Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning (2015, background): “Hence, the sequence of policies generated by the algorithm converges almost surely to a unique ﬁxed point π ∗ from Lemma 1 and the Contraction Mapping Theorem Bertsekas (1995).”
Reinforcement learning in robotics: A survey (2013, background): “In the limit when γ approaches 1, the metric approaches what is known as the averagereward criterion (Bertsekas, 1995),”
Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimation (2019, background): “ss for bias reduction is signicantly dierent, and instead yields an intriguing connection with the fundamental primal-dual relations between the density (ratio) functions and value functions (e.g., Bertsekas, 2000; Puterman, 2014). Our new perspective may allow us to motivate new algorithms for more”
The Uncertainty Bellman Equation and Exploration (2017, methodology): “s environment to maximize cumulative rewards through time [35]. We model the environment as a Markov decision process (MDP), but where the agent is initially uncertain of the true dynamics of the MDP [4, 5]. At each time-step, the agent performs an action, receives a reward, and moves to the next st”
Learning dexterous in-hand manipulation (2018, background): “For an in-depth introduction to RL, please refer to Sutton and Barto (1998) and Bertsekas (2005).”
Reinforcement Learning: A Survey (1996, background): “…model, in which the agent is supposed to take actions that optimize its long-run average reward: Such a policy is referred to as a gain optimal policy; it can be seen as the limiting case of the in(cid:12)nite-horizon discounted model as the discount factor approaches 1 (Bertsekas, 1995).”
Optimal and Approximate Q-value Functions for Decentralized POMDPs (2008, methodology): “It performs a guided A*-like search over partially specified joint policies, pruning joint policies that are guaranteed to be worse than the best (fully specified) joint policy found so far by an admissible heuristic.”
Noisy Networks for Exploration (2017, background): “MDPs model stochastic, discrete-time and ﬁnite action space control problems (Bellman & Kalaba, 1965; Bertsekas, 1995; Puterman, 1994).”
Infinite-Horizon Policy-Gradient Estimation (2001, methodology): “(20) ( Bertsekas, 1995 ). Hence, by (15)”