Boekbespreking van D.P. Bertsekas (ed.), Dynamic programming and optimal control - volume 2

By J. Nieuwenhuis. 1999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

Trust Region Policy Optimization (2015, methodology): “Most policy optimization methods can be classified into three broad categories: policy iteration methods, which alternate between estimating the value function under the current policy and updating the policy to choose better actions [1]; policy gradient methods, which use an estimator of the gradie”
High-Dimensional Continuous Control Using Generalized Advantage Estimation (2015, methodology): “1Another natural choice is to compute target values with an estimator based on the TD(λ) backup [2, 13], mirroring the expression we use for policy gradient estimation: V̂ λ t = Vφold (sn) + ∑∞ l=0(γλ) δt+l.”
Bridging the Gap Between Value and Policy Based Reinforcement Learning (2017, background): “We then have = max s F τ ( q (1) s ) − F τ ( q (2) s ) ≤ max The existence and uniqueness of V ∗ then follows from the contraction map ﬁxed-point theorem (Bertsekas, 1995).”
A unified view of entropy-regularized Markov decision processes (2017, methodology): “We now present a dynamic-programming view of the regularized optimization problem (Bertsekas, 2007) for the choice R = D C ( ·(cid:107) µ (cid:48) ) .”
Robot navigation in dense human crowds: Statistical models and experimental studies of human–robot cooperation (2015, background): “Specifically, they formalize robot motion planning in dynamic, uncertain environments as a stochastic dynamic program (see Bertsekas [12, 13]); intractability is avoided with receding horizon control techniques (Mayne et al.”
Dual Representations for Dynamic Programming (2008, methodology): “Then a standard LP for solving the planning problem can be expressed as (Puterman, 1994; Bertsekas, 1995; Bertsekas and Tsitsiklis, 1996) where v , an | S | × 1 vector, is the state value function.”
A Deep Learning Approach for Joint Video Frame and Reward Prediction in Atari Games (2016, background): “Optimal control theory describes mathematical principles for deriving control policies in continuous action spaces that maximize cumulative future reward in scenarios with known system dynamics and known reward structure (Bertsekas, 2007; 2005).”
A Finite Time Analysis of Temporal Difference Learning With Linear Function Approximation (2018, methodology): “…V µ = V µ , where the Bellman operator T µ associates a function V : S → R with another value function T µ V satisfying We assume rewards are bounded uniformly such that Under this assumption, value functions are assured to exist and are the unique solution to Bellman’s equation [Bertsekas, 2012].”
Construction of approximation spaces for reinforcement learning (2013, background): “This article will focus on the space of linear functions with p non-linear basis functions {φ i (·)} p i=1 (Bertsekas, 2007), which we call approximation space F φ .”
Experience Replay for Real-Time Reinforcement Learning Control (2012, background): “Such representations simplify the theoretical analysis of the resulting algorithms, see, e.g., [5] and Ch. 6 of [3].”
Optimal Management and Sizing of Energy Storage Under Dynamic Pricing for the Efficient Integration of Renewable Energy (2015, methodology): “The optimal policy and the cost-to-go functions can be computationally evaluated using standard dynamic programming techniques such as value/policy iteration or solving a linear program [20].”
Algorithms for Reinforcement Learning (2010, background): “That the sample is changed is essential if no good initial policy is available, i.e., when in the initial sample states which are frequently visited by “good” policies are underrepresented (a theoretical argument for why this is important is given by Van Roy (2006) in the context of state aggregatio”
Stable Dual Dynamic Programming (2007, methodology): “By the contractio n map fixed point theorem [2] there exists a unique fixed point of O in the space of vectors q.”
Neuro-Dynamic Programming (2009, background): “reinforcement learning, and are described in a number of sources, including the books by Bertsekas and Tsitsiklis (1996)”
Min-max approximate dynamic programming (2011, methodology): “…a discrete time-invariant dynamical system, with dynamics described by xt+1 = f(xt, ut, wt), t = 0, 1, . . . , (1) 978-1-4577-1067-4/11/$26.00 ©2011 IEEE 424 where xt ∈ X is the system state, ut ∈ U is the control input or action, wt ∈ W is an exogenous noise or disturbance, at time t, and f…”
Approximate dynamic programming via iterated Bellman inequalities (2015, methodology): “Bellman inequality Let O V W X ! R be a function that satisfies the Bellman inequality [2, 3], O V 6 T O V : (15)”