Dynamic Policy Programming with Function Approximation

By Mohammad Gheshlaghi Azar, Vicenç Gómez, Hilbert J. Kappen. 2011

39 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Trust-PCL: An Off-Policy Trust Region Method for Continuous Control (2017, methodology): “…previous work has used the relative entropy ( i.e., KL divergence) to regularize policy optimization; resulting in a softmax relationship between the optimal policy and state values (Peters et al., 2010; Azar et al., 2012, 2011; Fox et al., 2016; Rawlik et al., 2013) under single-step rollouts.”
Bridging the Gap Between Value and Policy Based Reinforcement Learning (2017, methodology): “Even though entropy regularized policy optimization is a well studied topic in RL [46, 39, 40, 47, 5, 4, 6, 7]–in fact, one that has been attracting renewed interest from concurrent work [25, 11]–we contribute new observations to this study that are essential for the methods we propose: first, we id”
A unified view of entropy-regularized Markov decision processes (2017, background): “Similar derivations have been done several times for discounted and episodic MDPs (Littman and Szepesvári, 1996; Ruszczyński, 2010; Azar et al., 2011; Rawlik et al., 2012; Asadi and Littman, 2016; Fox et al., 2016), but we are not aware of any work that considers the average-reward case.”
On the Use of Non-Stationary Policies for Stationary Infinite-Horizon Markov Decision Processes (2012, background): “Interestingly, this very constant 2γ (1−γ)2 appears in many works analyzing AVI algorithms [25, 11, 27, 12, 13, 23, 7, 6, 20, 21, 22, 9], API algorithms [15, 19, 16, 1, 8, 18, 5, 17, 10, 3, 9, 2] and in one of their generalization [26], suggesting that it cannot be improved.”
Non-parametric Policy Search with Limited Information Loss (2017, methodology): “For example, the dynamic policy programming algorithm (Azar et al., 2011) uses the relative entropy between successive policies as an additional cost term in the value function.”