An online primal-dual method for discounted Markov decision processes

By Mengdi Wang, Yichen Chen. 2016

25 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research, Reinforcement Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. Aviral Kumar
3. Ofir Nachum
4. George Tucker
5. Justin Fu
6. Shixiang Shane Gu
7. Chelsea Finn
8. Pieter Abbeel

In Management Science and Operations Research

1. Sergey Levine
2. Donghwan Lee
3. Aviral Kumar
4. Niao He
5. Justin Fu
6. Mengdi Wang
7. George Tucker
8. Yichen Chen

In Reinforcement Learning

1. Sergey Levine
2. Aviral Kumar
3. Ofir Nachum
4. George Tucker
5. Justin Fu
6. Shixiang Shane Gu
7. Chelsea Finn
8. Pieter Abbeel

In Optimization & Training

1. Donghwan Lee
2. Niao He
3. Mengdi Wang
4. Aviral Kumar
5. George Tucker
6. Justin Fu
7. Sergey Levine
8. Yichen Chen

Citations in the corpus

• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, background): “Lee and He (2018) extend a line of work applying to convex optimization techniques to policy optimization (Chen and Wang, 2016; Wang and Chen, 2016; Dai et al., 2017a,b) to the off-policy setting.”
• Stochastic Primal-Dual Q-Learning (2018, neutral): “In this subsection, we briefly review a linear programming (LP) formulation of the dynamic programming problem from Puterman (2014); Wang and Chen (2016); Chen and Wang (2016).”
• Stochastic Primal-Dual Methods and Sample Complexity of Reinforcement Learning (2016, background): “The closest prior work to the current one is (Chen and Wang, 2016), which proposed the first primal-dual-type learning method and a loose error bound.”
• Stochastic Primal-Dual Q-Learning Algorithm For Discounted MDPs (2019, methodology): “Recently, a stochastic primal-dual algorithm (SPD-RL) was proposed in [14], [15] to solve the convex-concave saddle-point problem in (3), which updates the primal and dual solutions simultaneously using noisy estimates of partial derivatives of the Lagrangian function obtained from samples of state-”
• Stabilizing Adversarial Nets With Prediction Methods (2017, methodology): “Similar optimization algorithms have been studied for reinforcement learning (Wang & Chen, 2016; Du et al., 2017).”
• Primal-Dual π Learning: Sample Complexity and Sublinear Run Time for Ergodic Markov Decision Problems (2017, background): “For the discounted MDP, the work [26] proposed a basic stochastic primal-dual iteration without explicit complexity analysis and later [6] ).”
• Diff-DAC: Distributed Actor-Critic for Multitask Deep Reinforcement Learning (2017, background): “Although there have been previous works on primal-dual formulations of RL (Wang, Bowling, and Schuurmans 2007; Valcarcel Macua et al. 2015; Wang and Chen 2016; Du et al. 2017), as well as an intuitive connection between actorcritic and bilevel methods (Pfau and Vinyals 2016), to the best of our…”
• Randomized Linear Programming Solves the Discounted Markov Decision Problem In Nearly-Linear (Sometimes Sublinear) Running Time (2017, background): “The work [29] and its journal version [8] considered a basic stochastic primal-dual iteration that uses Euclidean projection and uniform sampling of state-action pairs to solve DMDP and established a sample complexity upper bound O(|S||A| −2).”