Dynamic Programming and Optimal Control

By P. Bertsekas, Dimitri P. Bertsekas, Angelia Nedić, Asuman E. Ozdaglar. 2010

Subfields: Reinforcement Learning, Mathematical Optimization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Mathematical Optimization

Citations in the corpus

Value Iteration Networks (2016, background): “We assume that there is some unknown MDP ¯ M such that the optimal plan in ¯ M contains useful information about the optimal policy in the original task M .”
Deep learning in neural networks: An overview (2014, neutral): “Many variants of traditional RL exist (e.g., Abounadi, Bertsekas, & Borkar, 2002; Baird, 1995; Baird & Moore, 1999; Barto, Sutton, & Anderson, 1983; Bertsekas, 2001; Bradtke, Barto, & Kaelbling, 1996; Brafman & Tennenholtz, 2002; Kaelbling, Littman, & Cassandra, 1995; Lagoudakis & Parr, 2003; Maei…”
Variational Bayesian Reinforcement Learning with Regret Bounds (2018, background): “We consider the reinforcement learning problem, whereby an agent interacts with an environment in an episodic manner and attempts to maximize its return given the information it obtains from its interactions [102, 16], and where the environment is a Markov decision process (MDP) [84].”
Learning Interactive Real-World Simulators (2023, background): “Learning an accurate dynamics model in reaction to control inputs has been a long-standing challenge in system identification [50], model-based reinforcement learning [51], and optimal control [52, 53].”
Approximate Linear Programming for Average Cost MDPs (2013, background): “Hence, there are also bounded optimal solutions of LSALP( α ).”
Optimal control theory and the linear bellman equation (2011, unknown): “See Bertsekas (2000) for a further discussion.”
Efficiently Breaking the Curse of Horizon: Double Reinforcement Learning in Infinite-Horizon Processes (2019, methodology): “One can estimate the q -function by a value iteration in a ﬁnite-state-and-action-space setting utilizing an approximated MDP based on the empirical distribution (Bertsekas, 2012).”
Online Planning for Large Markov Decision Processes with Hierarchical Decomposition (2015, background): “If the discount factor γ = 1, the resulting MDP is then called undiscounted goal-directed MDP (a.k.a. stochastic shortest path problem [Bertsekas 1996]).”
Linear Programming for Large-Scale Markov Decision Problems (2014, unknown): “For a survey on theoretical results for ADP see (Bertsekas and Tsitsiklis, 1996, Sutton and Barto, 1998), (Bertsekas, 2007, Vol. 2, Chapter 6), and more recent papers (Sutton et al., 2009b,a, Maei et al., 2009, 2010).”
Energy Management in Plug-in Hybrid Electric Vehicles: Recent Progress and a Connected Vehicles Perspective (2017, methodology): “This cost function J for DP deterministic implementation (DDP) can be expressed as [3], [32]”
Efficient High-Dimensional Stochastic Optimal Motion Control using Tensor-Train Decomposition (2015, methodology): “Once a consistent discretization is obtained, the direct method is to utilize the standard dynamic programming algorithms, such as value iteration or policy iteration [2].”
End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks (2019, methodology): “Most policy optimization RL algorithms attempt to maximize long-term reward J ( π ) using (a) policy iteration meth-ods (Bertsekas 2005), (b) derivative-free optimization meth-ods that optimize the return as a function of policy parameters (Fu, Glover, and April 2005), or (c) policy gradient…”
Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach (2015, methodology): “We overcome the aforementioned computational challenges (due to the continuous augmented state) by designing an algorithm that merges approximate value iteration [3] with linear interpolation.”
Revenue Management with Forward-Looking Buyers (2016, background): “When there is a single house and valuations are independently and identically distributed (IID), the cutoff value is constant, except for the last period (see Bertsekas 2005, 185). Ross (1971) studies this problem directly in continuous time. McAfee and McMillan (1988) introduce private information ”
Dynamic modeling and control of supply chain systems: A review (2008, methodology): “For sake of completeness, we will briefly describe the dynamic programming philosophy, in the discrete-time, finitehorizon setting [9].”
Update or wait: How to keep your data fresh (2016, unknown): “This tells us that there exists an optimal policy π opt = (Z 0 , Z 1 , . . .) ∈ Π of Problem (4) in which the control action Z i is determined based on only Y i , but not the history state and control Y 0 , . . . , Y i−1 , Z 0 , . . . , Z i−1 [16] .”