Dual Representations for Dynamic Programming

By Tao Wang, Daniel J. Lizotte, Michael H. Bowling, Dale Schuurmans. 2008

14 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Information Systems, Management Science and Operations Research, Algorithms & Complexity

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Information Systems

In Management Science and Operations Research

In Algorithms & Complexity

Citations in the corpus

Reinforcement Learning via Fenchel-Rockafellar Duality (2020, methodology): “Is there a better way? There is a more direct way to frame the policy optimization problem as a convex problem, and it is given by the V -LP (Puterman, 1994; Bertsekas et al., 1995; Bertsekas and Tsitsiklis, 1996; Wang et al., 2007).”
AlgaeDICE: Policy Gradient from Arbitrary Experience (2019, background): “This objective may be equivalently written in its dual form (Puterman, 1994; Wang et al., 2008) in terms of the policy’s normalized state visitation distribution as maxπ JD(π) := E(s,a)∼dπ [r(s, a)] , (3)”
Off-Policy Deep Reinforcement Learning by Bootstrapping the Covariate Shift (2019, neutral): “This is analogous to the perspective of the discount factor as a probability of terminating (White 2017), and is related to the constraint that arises in the dual formulation of the value function (Wang et al. 2008).”
Boosting the Actor with Dual Critic (2017, methodology): “…representation of Bellman optimality equation and its duality have been utilized for (approximate) planning problem (de Farias and Roy, 2004; Wang et al., 2008; Pazis and Parr, 2011; O’Donoghue et al., 2011; Malek et al., 2014; Cogill, 2015), in which the transition probability of the MDP…”
A Distributional Perspective on Reinforcement Learning (2017, background): “Wang et al. (2008) considered the dual formulation of re-inforcement learning, where one optimizes the stationary distribution subject to constraints given by the transition function (Puterman, 1994), in particular its relationship to linear approximation.”
Linear Programming for Large-Scale Markov Decision Problems (2014, unknown): “Wang et al. (2008) study ALP (3) and show that there is a dual form for standard value function based algorithms, including on-policy and oﬀ-policy updating and policy improvement.”
Unifying Count-Based Exploration and Intrinsic Motivation (2016, methodology): “method (Veness et al., 2015) applies sequential density models to the problem of reinforcement learning. It uses these models to learn a form of dual value function based on stationary distributions (Wang et al., 2008). In the Markov Decision Process (MDP) setting, a policy ˇis a mapping from states”