Stochastic optimal control : the discrete time case

By Dimitri P. Bertsekas, Steven E. Shreve. 2007

1,637 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Continuous Doubly Constrained Batch Reinforcement Learning (2021, background): “TCDC is a L∞ contraction under standard conditions that suffice for the ordinary Bellman operator to be contractive [3, 6, 8, 56].”
Least Squares Temporal Difference Methods: An Analysis under General Conditions (2012, background): “(See, e.g., Bertsekas and Shreve [6] for general space MDP theory.)”
Risk-averse dynamic programming for Markov decision processes (2010, background): “Several excellent textbooks and monographs [5, 6, 21, 27, 32, 39, 50] discuss various aspects of this vast area.”
Reinforcement learning for control: Performance, stability, and deep approximators (2018, background): “An important property of MDPs is that, under appropriate conditions, there exists a deterministic optimal policy that maximizes the value, despite the fact that transitions are stochastic (Bertsekas and Shreve, 1978).”
Comparison of some control strategies for three-compartment PK/PD models (1994, background): “In this context, the above stated principle of optimality still holds if we replace the costs by their expectations, taken with respect to the future observation and the prior probability distribution of the parameters ( 18 ).”
Impulse control problem on finite horizon with execution delay (2007, unknown): “By a measurable selection theorem (see e.g. Chapter 7 in [6]), there exists ¯ α ε ∈ A θ,p ( θ,α ) s.t. ¯ α ε ( ω ) = α ε,ω ( ω ) a.s., and so Now, we deﬁne by concatenation the impulse control ¯ α consisting of the impulse control components of α until (including eventually) time τ , and the…”
Algorithms for Reinforcement Learning (2010, background): “The reader who is interested in learning more about MDPs is suggested to consult one of the many excellent books on the subject, such as the books of Bertsekas and Shreve (1978), Puterman (1994), or the two-volume book by Bertsekas (2007a,b).”
Arbitrage and duality in nondominated discrete-time models (2013, background): “ P(ω)[dR/dP(ω)] +ER[dR/dRˆ ], where dR/dP(ω) and dR/dRˆ are the jointly Borel-measurable Radon–Nikodym derivatives as in Lemma 4.7. Then P(ω) ≪ R≪ Rˆ if and only if φ(ω,R,Rˆ) = 2. Since Lemma 4.7 and [6], Proposition 7.26, page 134, imply that φis Borel, we deduce that graph(Φ)= {φ=2}⊆Ωt ×P(Ω1) is B”
Approximate Dynamic Programming for Ambulance Redeployment (2010, background): “For basic terminology, notation, and fundamental results regarding dynamic programming in uncountable state spaces, we refer the reader to Bertsekas and Shreve (1978).”
Modeling of risk-based inspection, maintenance and life-cycle cost with partially observable Markov decision processes (2005, background): “…management, machine maintenance, reservoir control, structural inspection and rehabilitation, and the control of physical systems subject to random disturbances (Bellman 1957, Blackwell 1965, Dynkin 1965, Aoki 1967, Martin 1967, Sondik 1971, Bertsekas and Shreve 1978, Hee 1978, Monahan 1982).”
Optimality Conditions for Inventory Control (2016, methodology): “This reduction holds for problems with Borel state, action, and observation sets, as well as with measurable transition probabilities (Bertsekas and Shreve [12], HernándezLerma [38], Rhenius [48], Yushkevich [65]).”
Optimal arbitrage under model uncertainty. (2011, background): “…functional α : [0,∞)×Ω→ Sn which satisfies α(t,ω) ∈A(ω(t)) for all (t,ω) ∈ [0,∞)×Ω,(7.12) we can select a progressively measurable functional ϑ : [0,∞)×Ω→Rn with (ϑ(t,ω), α(t,ω)) ∈K(ω(t)) ∀(t,ω) ∈ [0,∞)×Ω(7.13) [see the “measurable selection” results in Chapter 7 of Bertsekas and Shreve (1978)].”