On the Convergence of Policy Iteration in Stationary Dynamic Programming

By Martin L. Puterman, Shelby L. Brumelle. 1979

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Romain Laroche
2. Peter A. Forsyth
3. George Labahn
4. Paul Trichelair
5. Michael L. Littman
6. Sergey Levine
7. Aviral Kumar
8. Justin Fu

Citations in the corpus

• Safe Policy Improvement with Baseline Bootstrapping (2017, methodology): “In practice, the optimisation process may be performed by policy iteration [11, 25]: the current policy π is evaluated with Q, and then the next iteration policy π is made greedy with respect to Q under the constraint of belonging to Πb (see Algorithm 4 in Appendix, Section D).”
• Algorithms for Sequential Decision Making (1996, background): “Theorems related to Theorem B.2 are presented by Puterman and Brumelle [124,125] in a more abstract setting that might make it possible to prove superlinear con-vergence of policy iteration, given some additional analysis.”
• Numerical methods for controlled Hamilton-Jacobi-Bellman PDEs in finance (2007, background): “Nevertheless, as discussed in [40, 39, 42], we may view iteration (6.”