Reinforcement learning with Gaussian processes

By Yaakov Engel, Shie Mannor, R. Meir. 2005

430 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Reinforcement Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Reinforcement Learning

In Probabilistic & Bayesian Methods

Citations in the corpus

Modelling transition dynamics in MDPs with RKHS embeddings (2012, methodology): “Kernel methods have become increasingly popular in RL. Methods include kernel LSTD (Xu et al., 2005) and GPTD (Engel et al., 2005).”
Informing sequential clinical decision-making through reinforcement learning: an empirical study (2010, background): “In the reinforcement learning literature, Bayesian approaches have also been explored for similar reasons (Dearden et al. 1999 ;S trens2000 ;W ang et al.2005; Engel et al. 2005 ); other methods targeting efficient exploration have also been explored (Kakade et al. 2003; Strehl et al. 2006; Brunskil”
Distributional Reinforcement Learning with Quantile Regression (2017, background): “…that the value distribution is not designed to capture the uncertainty in the estimate of the value function (Dear-den, Friedman, and Russell 1998; Engel, Mannor, and Meir 2005), that is the parametric uncertainty , but rather the randomness in the returns intrinsic to the MDP. Temporal…”
A Distributional Perspective on Reinforcement Learning (2017, methodology): “Engel et al. (2005) leveraged the distributional Bellman equation to deﬁne a Gaussian process over the unknown value function.”
Policy Networks with Two-Stage Training for Dialogue Systems (2016, methodology): “Consequently, the value functions as well as the policy of the agent are computed on the belief states B t : (3) In this paper, we use GPSARSA as a baseline as it has been proved to be a successful algorithm for training POMDP-based dialogue managers (Engel et al., 2005; Gaˇsi´c et al., 2010).”
BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems (2016, methodology): “A recent paper (Fatemi et al. 2016) compares deep RL (both policy gradient and Q-learning) to GP-SARSA (Engel, Mannor, and Meir 2005) on a simpler dialogue policy learning problem.”
Active Reward Learning (2014, methodology): “[12] has proposed to model the value function through a GP and Ghavamzadeh and Engel [13] has proposed to model the policy gradient through a GP.”
A Non-Parametric Approach to Dynamic Programming (2011, methodology): “Methods such as kernelized least-squares temporal difference learning [24] and Gaussian process temporal difference learning [23] have also employed kernels in policy evaluation.”
Efficient Exploration for Dialog Policy Learning with Deep BBQ Networks \& Replay Buffer Spiking (2016, methodology): “A number of papers explore Gaussian processes for learning dialog policies [Gaˇsi´c et al., 2010, Fatemi et al., 2016], using the GP-SARSA algorithm [Engel et al., 2005].”
Interval Estimation for Reinforcement-Learning Algorithms in Continuous-State Domains (2010, background): “Bayesian methods (e.g., GPTD [6]) provide a natural measure of confidence: one can use the posterior distribution to form credible intervals for the mean value of a state-action pair.”
Sample Efficient Reinforcement Learning with Gaussian Processes (2014, methodology): “We show that existing model-free algorithms (Engel et al., 2005; Chung et al., 2013) that use a single GP may require an exponential (in the discount factor) number of samples to reach a near-optimal policy.”
Bayesian Reinforcement Learning (2010, methodology): “Finally, the GPTD model can be used to derive a SARSA-type algorithm, called GPSARSA (Engel et al, 2005a; Engel, 2005), in which state-action values are estimated using GPTD and policies are improved by a ε -greedily strategy while slowly decreasing ε toward 0.”
On-line policy optimisation of spoken dialogue systems via live interaction with human subjects (2011, methodology): “GP-Sarsa is an on-line RL algorithm that models the Q-function as a zero mean Gaussian process [17] which defines correlations in”