Policy Gradient in Continuous Time

By R´emi Munos. 2006

96 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Optimization & Training

Citations in the corpus

Learning Continuous Control Policies by Stochastic Value Gradients (2015, methodology): “Our example is based on a problem due to [16].”
Gradient Estimation Using Stochastic Computation Graphs (2015, background): “What we call the pathwise derivative estimator (from the mathematical finance literature [4] and reinforcement learning [16]) is alternatively called infinitesimal perturbation analysis and stochastic backpropagation [21].”
Policy gradients in linearly-solvable MDPs (2010, methodology): “An RL method for estimating Ow in deterministic problems was developed in [9].”
Parameter-exploring policy gradients (2010, unknown): “Furthermore , the variance increases linearly with the length of the history Munos and Littman (2006), since each state depends on the entire sequence of previous samples.”
Monte Carlo Gradient Estimation in Machine Learning (2019, background): “sed estimators in such settings. We defer to the papers by Fournie et al. (1999) and Benhamou (2003) in computational nance, and, closer to machine learning, the papers by Gobet and Munos (2005) and Munos (2006) in optimal control and reinforcement learning for further exploration of these ideas. 9.”
Non-parametric policy gradients: a unified treatment of propositional and relational domains (2008, methodology): “Within policy-based methods, policy gradients have received increased attention, see e.g. (Williams, 1992; Baxter et al., 2001; Guestrin et al., 2002; Konda & Tsitsiklis, 2003; Munos, 2006) as a non-exhausting list.”
Receding Horizon Differential Dynamic Programming (2007, methodology): “Local methods have also been used for purely policy-based algorithms [8, 9, 10], without explicit representation of the value function.”
Value-gradient learning (2012, methodology): “Alternatively, there is an extremely fast on-line model-learning method by [20] which could be used.”
Exploring parameter space in reinforcement learning (2010, background): “Furthermore, the variance increases linearly with the length of the history [12], since each state may depend on the entire sequence of previous samples.”
Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEs (2020, methodology): “…decision processes (SMDPs) [Howard, 1964]—including Q -learning [Bradtke and Duff, 1995], advantage updating [Baird, 1994], policy gradient [Munos, 2006], actor-critic [Doya, 2000]—extend the standard RL framework to continuous time, but all use relatively simple linear function…”
Learning to Control a 6-Degree-of-Freedom Walking Robot (2007, background): “Hence, the agent’s exploratory behavior does not give any clues to policy improvement [6].”