Stable Dual Dynamic Programming

By Tao Wang, Daniel J. Lizotte, Michael H. Bowling, Dale Schuurmans. 2007

35 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Sergey Levine
2. Ofir Nachum
3. Yuxi Li
4. Yoshua Bengio
5. Bo Dai
6. Pieter Abbeel
7. Lihong Li
8. J. Urgen Schmidhuber

Citations in the corpus

• Black-box Off-policy Estimation for Infinite-Horizon Reinforcement Learning (2020, background): “Note that similar forms of B π have appeared in the literature, usually used to encode constraints in a dual linear program for an MDP [4,37,38].”
• GradientDICE: Rethinking Generalized Offline Estimation of Stationary Values (2020, methodology): “Similar to Zhang et al. (2020a), we consider two performance metrics for the policy π: the total discounted reward ργ(π) .”
• Compress and Control (2014, background): “proximate a type of dual representation (Wang, Bowling, and Schuurmans 2007; Wang et al. 2008) of the value function.”
• Provably Convergent Two-Timescale Off-Policy Actor-Critic with Function Approximation (2019, methodology): “…algorithm, Gradient Emphasis Learning (GEM), to approximate the emphasis in COF-PAC, inspired by Gradient TD methods (GTD, Sutton et al. 2009b;a), Emphatic TD methods (ETD, Sutton et al. 2016), and reversed TD methods (Wang et al., 2007; 2008; Hallak & Mannor, 2017; Gelada & Bellemare, 2019).”
• Scalable Bilinear π Learning Using State and Action Features (2018, methodology): “Previous work also uses linear approximation of value functions (Bertsekas and Tsitsiklis, 1996; Elkan, 2011), or state transition probabilities (Wang et al., 2007; Yang et al., 2009).”
• Apprenticeship learning using linear programming (2008, methodology): “In particular, the dual linear programming approach in this work might com­bine nicely with recent work on stable MDP approximation techniques based on the dual form (Wang et al., 2008).”
• Dynamic Policy Programming with Function Approximation (2011, background): “Wang et al. (2007) introduce a dual representation technique called dual dynamic programming (dual- 4These basis functions are randomly placed within the range of their inputs.”
• Algorithms for Reinforcement Learning (2010, methodology): “Other topics of interest include the linear programming-based approaches (de Farias and Van Roy, 2003, 2004, 2006), dual dynamic programming (Wang et al., 2008), techniques based on sample average approximation (Shapiro, 2003) such as PEGASUS (Ng and Jordan, 2000), online learning in MDPs with…”
• Dual Temporal Difference Learning (2009, background): “In particular, the dual representation provides an equivalent but distinct approach to solving DP and RL problems (Wang et al. 2007, 2008).”