Kernelized value function approximation for reinforcement learning

By Gavin Taylor, Ronald E. Parr. 2009

127 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Control and Systems Engineering

Citations in the corpus

Modelling transition dynamics in MDPs with RKHS embeddings (2012, background): “Both can be used to estimate (q)values and they diﬀer in this mainly through the regulariser (Taylor & Parr, 2009).”
A Kernel Loss for Solving the Bellman Equation (2019, methodology): “In RL, authors have used kernels either to smooth the estimates of transition probabilities and rewards (Ormoneit & Sen, 2002), or to represent the value function (e.g., Xu et al., 2005, 2007; Taylor & Parr, 2009).”
Non-Parametric Approximate Linear Programming for MDPs (2011, methodology): “Other non-parametric approaches to reinforcement learning include variable resolution grids (Munos and Moore 2002), the work of Ormoneit and Sen (2002), Fitted Q- Iteration with tree-based approximators (Ernst, Geurts, and Wehenkel 2005) and many kernelized methods (Taylor and Parr 2009).”
Non-parametric Policy Search with Limited Information Loss (2017, methodology): “…proposed for (partially observable) MDPs (Grünewälder et al., 2012b; Nishiyama et al., 2012), and non-parametric approximate dynamic programming method have been proposed by Ormoneit and Sen (2002), Taylor and Parr (2009), Deisenroth et al. (2009), Kroemer and Peters (2011) and Xu et al. (2014).”
Proximal Reinforcement Learning: A New Theory of Sequential Decision Making in Primal-Dual Spaces (2014, background): “Future research is ongoing to explore other operator splitting techniques beyond primal-dual splitting as well as incorporating random projections [113], and investigating kernelized algorithms [114, 115].”
Regularized Policy Iteration with Nonparametric Function Spaces (2016, methodology): “Mapping P evaluated Farahmand, Ghavamzadeh, Szepesvári, and Mannor at (x, a) ∈ X × A gives a distribution over R × X , which we shall denote by P (·, ·|x, a).”
A Non-Parametric Approach to Dynamic Programming (2011, methodology): “As a result, KTD usually needs to be regularized [15].”
Real-Time Bidding by Reinforcement Learning in Display Advertising (2017, background): “For large-scale situations, it is difficult to experience the whole state space, which leads to the use of function approximation that constructs an approximator of the entire function [8, 22].”
Policy evaluation with temporal differences: a survey and comparison (2015, methodology): “This approach, KLSTD and GPTD were unified in a model-based framework for kernelized value function approximation by Taylor and Parr (2009).”
Learning from Limited Demonstrations (2013, methodology): “Most methods attempt to exploit the structures in the value function [8, 9, 10, 11], but in some problems one might have extra information about the structure of good or optimal policies as well.”
Algorithms for Reinforcement Learning (2010, methodology): “Examples of this line of research include the use of regression trees (Ernst et al., 2005), or “kernelizing” the value estimation algorithms (e.g., Rasmussen and Kuss, 2004; Engel et al., 2005; Ghavamzadeh and Engel, 2007; Xu et al., 2007; Taylor and Parr, 2009).”
Manifold Regularization for Kernelized LSTD (2017, background): “The two-room navigation problem is a classic benchmark for RL algorithms that cope with either discrete [16, 17, 15] or continuous state space [14, 18].”
Error Propagation for Approximate Policy and Value Iteration (2010, background): “The MDP is a 100-state, 2-action chain similar to Chain Walk problem in the work of Lagoudakis and Parr [5].”