Approximate dynamic programming: solving the curses of dimensionality

By P. Pardalos. 2009

1,779 citations recorded by the source corpus.

Subfields: Management Science and Operations Research, Reinforcement Learning, Mathematical Optimization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

In Reinforcement Learning

In Mathematical Optimization

Citations in the corpus

Safe Model-based Reinforcement Learning with Stability Guarantees (2017, methodology): “In our experiments, we use approximate dynamic programming [36] to capture the performance of the policy.”
Variational Bayesian Reinforcement Learning with Regret Bounds (2018, methodology): “This is a dimensionality reduction and generalization strategy that, in the no uncertainty case, has theoretical guarantees [25, 82], and has been used successfully in continuous control [109, 110, 66, 67, 60].”
Generalization and Exploration via Randomized Value Functions (2014, methodology): “Algorithm 1 Least-Squares Value Iteration Input: Φ0, . . . ,ΦH−1 ∈ |S||A|×K , λ ∈ ++, {(xih, aih, rih) : i j, h = 0, . . . ,H − 1} Output: θj0, . . . , θj,H−1 1: θjH ← 0, ΦH ← 0 2: for h = H − 1, . . . , 1, 0 do 3: Generate regression matrix and vector A← Φh(x0h, a0h)... Φh(xj−1,h, aj−1,h) b←…”
Compress and Control (2014, background): “(Howard 1960), approximate dynamic programming (Bertsekas and Tsitsiklis 1996; Powell 2011) and actor-critic methods (Sutton and Barto 1998), for example, all crucially depend on how well policy evaluation can be performed.”
The endogenous grid method for discrete‐continuous dynamic choice models with (or without) taste shocks (2017, background): “9Referred to as the post-decision state in the operations research literature, Powell (2007). all discrete labor market choices dt can be calculated directly using the inverted Euler equation ct(A⃗t|dt) = (u′)−1 ( βrhs ( Mt+1(A⃗t) )) , (17) where rhs ( Mt+1(A⃗t) ) is the right hand side of (15)…”
A Function Approximation Method for Model-based High-Dimensional Inverse Reinforcement Learning (2017, neutral): “A good introduction is given in [6].”
A modern Bayesian look at the multi-armed bandit (2010, methodology): “Powell [12] notes that ‘Unfortunately, at the time of this writing, there do not exist easy to use software utilities for computing standard Gittins indices’.”
Finding multiple optimal solutions to optimal load distribution problem in hydropower plant (2012, neutral): “We can also try approximate dynamic programming [38] to approximate fi(si).”
AI, OR and Control Theory: A Rosetta Stone for Stochastic Optimization (2012, background): “In Powell (2011), I argue that a good definition for a state variable is Definition 3.1 A state variable is the minimally dimensioned function of history that is necessary and sufficient to compute the decision function, the transition function, and the contribution function.”
Approximate Dynamic Programming for Ambulance Redeployment (2010, methodology): “A generic approach for ADP involves using value function approximations of the form P p=1 r p φ p (·), where {r p : p = 1, . . . , P } are tunable parameters and {φ p (·) : p = 1, . . . , P } are fixed basis functions; see Bertsekas and Tsitsiklis (1996) and Powell (2007).”
Neural Adaptive Video Streaming with Pensieve (2017, methodology): “, Post-Decision States [6, 34]) could be used to accelerate learning in Pensieve as well.”
A review of dynamic vehicle routing problems (2013, methodology): “Powell et al. [113] formulated a truckload PDP as a Markov Decision Process (MDP).”
Pathologies of temporal difference methods in approximate dynamic programming (2010, methodology): “Note that even in this more general case, the policy evaluation Eq. (6) can be solved by using simulation and low order calculations (see [ Ber10a ] and the survey [Ber10b], which contains many references).”
On the Relationship Between Feature Selection and Classification Accuracy (2008, background): “In the literature, this phenomenon is referred to as the curse of dimensionality (Powell, 2007).”
An Intelligent Battery Controller Using Bias-Corrected Q-learning (2012, neutral): “The effect of this particular bias in applications has been observed earlier in estimating asset values in finance (Brown 1974), and by authors in reinforcement learning (Thurn and Schwartz 1993; Ormoneit and Sen 2002), management science (Smith and Winkler 2006) and in operations research (Powell 2”
Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data (2011, methodology): “RL methods have been developed starting from learning mechanisms observed in mammals [4], [7]–[9], [24], [28], [29], [33].”
Multistage Stochastic Programming: A Scenario Tree Based Approach to Planning under Uncertainty (2011, background): “to be taken into account — seems to be growing in the machine learning community, as indicated by a series of advances in approximate dynamic programming motivated by such applications (Powell, 2007; Csáji & Monostori, 2008).”
Approximate policy iteration: a survey and some new methods (2011, methodology): “We refer to the literature, and particularly the books [5, 7, 9] for an explanation of these terms and further discussion.”