On the existence of fixed points for approximate value iteration and temporal-difference learning
By D. Farias, Benjamin Van Roy. 2000
100 citations recorded by the source corpus.
Subfields: Control and Systems Engineering, Numerical Analysis
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Control and Systems Engineering
- Sergey Levine
- Abhishek Gupta
- Aviral Kumar
- Chelsea Finn
- Pieter Abbeel
- Doina Precup
- Daniel J. Lizotte
- Dale Schuurmans
In Numerical Analysis
- Vicenç Gómez
- Daniel J. Lizotte
- Dale Schuurmans
- Sergey Levine
- Hilbert J. Kappen
- Geoffrey J. Gordon
- Mohammad Gheshlaghi Azar
Citations in the corpus
- DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correction (2020, background): “We refer the reader to (Farias and Roy, 2000) for further reference.”
- Dynamic policy programming (2010, neutral): “Examples of such approximate dynamic programming (ADP) methods are approximate policy iteration (API) and approximate value iteration (AVI) (Bertsekas, 2007; Lagoudakis and Parr, 2003; Perkins and Precup, 2002; de Farias and Roy, 2000).”
- An analysis of reinforcement learning with function approximation (2008, background): “(t)+ b. where A. = E. fT .fy - fx ; b. = E. fT r(x, a, y) . xx An equilibrium point of (13) must verify .* = A-1b.* .* and the existence of such equilibrium point has been established in (de Farias & Van Roy, 2000) (Theorem 5.1).”
- Dynamic Policy Programming with Function Approximation (2011, methodology): “01), we also implement OAPI with a soft-max policy, since the existence of fixed point is guaranteed in this case (Farias and Roy, 2000).”
- Reinforcement Learning with Function Approximation Converges to a Region (2000, result): “Results similar to the ones presented here were developed independently in [6].”
- Stable Dual Dynamic Programming (2007, background): “We focus on the on-policy case here, because, famously, the max operator does not always have a fixed point when combined with approximation in the primal case [8], and consequently suffers the risk of divergence [5, 6].”
- A Convergent Form of Approximate Policy Iteration (2002, background): “De Farias and Van Roy [5] showed that a form of approximate value iteration which relies on linear value function approximations and softmax policy improvement is guaranteed to possess fixed points.”