Adaptive stepsizes for recursive estimation with applications in approximate dynamic programming

By Abraham P. George, Warrren B Powell. 2006

190 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research, Reinforcement Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Reinforcement Learning

In Optimization & Training

Citations in the corpus

Cyclical Learning Rates for Training Neural Networks (2015, background): “A review of the early work on adaptive learning rates can be found in George and Powell [6].”
Algorithms for Reinforcement Learning (2010, methodology): “A non-exhaustive list includes the use of adaptive step-sizes (Sutton, 1992; George and Powell, 2006), normalizing the updates (Bradtke, 1994) or reusing previous samples (Lin, 1992).”
Adaptive learning rates and parallelization for stochastic, sparse, non-smooth gradients (2013, background): “There are a number of adaptive settings for SGD learning rates, or equivalently, diagonal preconditioning schemes, to be found in the literature, e.g., [2, 3, 4, 5, 6, 7].”
Autonomous reinforcement learning with experience replay. (2013, methodology): “Consequently, the solution to the problem of step-size determination in reinforcement learning is still a subject of active research (George & Powell, 2006; Noda, 2009; Schraudolph, Yu, & Aberdeen, 2006).”
Optimal Hour-Ahead Bidding in the Real-Time Electricity Market with Battery Storage Using Approximate Dynamic Programming (2014, methodology): “After some experimentation, we found that the biasadjusted Kalman Filter (BAKF) developed in George and Powell (2006), performed better than simpler alternatives.”
Stochastic Gradient Descent: Going As Fast As Possible But Not Faster (2017, background): “Learning rate adaptation The dynamic adjustment of the learning rate was acknowledged a key issue since the early SGD days (see [9] for a review).”
An Overview of Stochastic Approximation (2015, background): “See [18] for an extensive review of both deterministic and stochastic step sizes.”
Probabilistic Line Searches for Stochastic Optimization (2015, background): “The former issue, adapting the search direction, has been addressed by many authors (an overview can be found in George & Powell, 2006).”
No More Pesky Learning Rate Guessing Games (2015, background): “A review of the early work on adaptive learning rates can be found in George and Powell [6].”
Value-Difference Based Exploration: Adaptive Control between Epsilon-Greedy and Softmax (2011, methodology): “For this, the branch of temporal-diﬀerence learning oﬀers two commonly used algorithms which are namely Sarsa for on-policy control [11]: and Q -learning for oﬀ-policy control [2]: where α is a stepsize parameter [12].”
A Stochastic Line Search Method with Convergence Rate Analysis (2018, background): “Various authors have attempted to address this last issue, see [8, 10, 12, 13].”