The Asymptotic Convergence-rate of Q-learning

By Csaba Szepesv. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Sergey Levine
2. Aviral Kumar
3. Justin Fu
4. Ofir Nachum
5. Ilya Kostrikov
6. Shixiang Shane Gu
7. Pieter Abbeel
8. Michael Janner

Citations in the corpus

• Diagnosing Bottlenecks in Deep Q-learning Algorithms (2019, methodology): “Online Q-learning methods can be viewed as a form of stochastic approximation (such as Robbins-Monro) applied to Q-iteration and FQI (Bertsekas & Tsitsiklis, 1996), and share many of its theoretical properties (Szepesvári, 1998).”
• Bias in Natural Actor-Critic Algorithms (2014, background): “Smaller values of γ are known to result in faster convergence of value function estimates (Szepesvari, 1997), however larger γ typically result in smoother value functions that may be easier to approximate accurately with few features.”