Gaussian process dynamic programming

By Marc Peter Deisenroth, Carl E. Rasmussen, Jan Peters. 2009

216 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Reinforcement Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Reinforcement Learning

In Probabilistic & Bayesian Methods

Citations in the corpus

Modelling transition dynamics in MDPs with RKHS embeddings (2012, methodology): “We consider the under-actuated pendulum swing up task (Deisenroth et al., 2009).”
Conditional mean embeddings as regressors (2012, methodology): “The reinforcement learning task is the under-actuated pendulum swing-up problem from Deisenroth et al. (2009).”
Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning (2011, methodology): “Although the approaches in [20, 8] do long-term planning for finding a policy, they cannot directly deal with constraints in the state space (e.”
Gaussian Processes for Data-Efficient Learning in Robotics and Control (2015, positive): “Instead closed-form Bayesian averaging over inﬁnitely many plausible dynamics models is possible by using non-parametric GPs. Nonparametric GP dynamics models in RL were previously proposed in [17], [30], [46], where the GP training data were obtained from “motor babbling”.”
Learning Non-stationary System Dynamics Online Using Gaussian Processes (2010, background): “[8] learned control policies of a completely unknown system in a GP framework.”
Reinforcement learning (2019, unknown): “[1] used GPs for approximate dynamic programming in reinforcement learning, as probabilistic function approximators for the value function, and as models of the system dynamics.”
Multi-modal estimation with kernel embeddings for learning motion models (2013, background): “As such they are capable of modelling complex relationships between variables with Gaussian marginals, and have been used for modelling terrain and occupancy [21], [22], optimal control [23], and for planning based on information gain [24].”
Probabilistic Differential Dynamic Programming (2014, background): “For instance, the works by Rasmussen et al.[10], Nguyen-Tuong et al.[11], Deisenroth et al.[12][13][14] and Hemakumara et al.[15] have demonstrated the remarkable applicability of GP-based control and RL methods in robotics.”
Reinforcement learning of motor skills in high dimensions: A path integral approach (2010, background): “…from statistical learning and empirical inference with classical RL approaches in order to minimize tuning parameters and numerical problems, such that ultimately more efﬁcient algorithms can be developed that scale to signiﬁcantly more complex learning systems [3], [4], [5], [6], [7], [8], [9].”
A Practical and Conceptual Framework for Learning in Control (2010, methodology): “Doya (2000) applied both model-free and model-based algorithms to learn the cart-pole task. The value function was modeled by an RBF network. If the dynamics model was learned (using a different RBF network), Doya (2000) showed that the resulting algorithm was more efficient than the model-free lear”
Fractional Neuro-Sequential ARFIMA-LSTM for Financial Market Forecasting (2020, background): “Peters [8] noted that the dynamic nature of the stock market which is mostly non-Gaussian in nature with sharper peaks and fat tails [9].”
Learning to soar: Resource-constrained exploration in reinforcement learning (2015, background): “Dearden et al. (1998) maintained probability distributions over the value function, while Engel et al. (2003, 2005) demonstrated the use of Gaussian process (GP) modeling for value function approximation and noted that the GP covariance could provide confidence bounds on the value estimate to…”
A Survey on Policy Search for Robotics (2013, background): “For instance, the value function of the under-powered pendulum swing-up is discontinuous along the manifold where the applicable torque is just not sufficient to swing the pendulum up [23].”
Nonlinear Inverse Reinforcement Learning with Gaussian Processes (2011, methodology): “GP regression requires input-output pairs [11], and was previously used for value function approximation [10, 4, 2].”
A Generalized Path Integral Control Approach to Reinforcement Learning (2010, unknown): “Equation uL(τ i) = R−1g (c) ti g (c)T ti R−1g (c) ti ( g (c)T ti ti − bti ) , on page (3145) should change to ∗.”
Principles of sensorimotor learning (2011, background): “For example, fast learning in the cart and pendulum task can be achieved by learning a probabilistic model of the input–output relationship so that it is possible to predict not only the expected response of the system to an action but also the confidence in all possible outcome”
PILCO: A Model-Based and Data-Efficient Approach to Policy Search (2011, methodology): “Algorithms with GP dynamics models in RL were presented by Rasmussen & Kuss (2004), Ko et al. (2007), and Deisenroth et al. (2009).”