Manifold Regularization for Kernelized LSTD
By Xinyan Yan, Krzysztof Marcin Choromanski, Byron Boots, Vikas Sindhwani. 2017
Policy evaluation or value function or Q-function approximation is a key procedure in reinforcement learning (RL). It is a necessary component of policy iteration and can be used for variance reduction in policy gradient methods. Therefore its quality has a significant impact on most RL algorithms. Motivated by manifold regularized learning, we propose a novel kernelized policy evaluation method that takes advantage of the intrinsic geometry of the state space learned from data, in order to achieve better sample efficiency and higher accuracy in Q-function approximation. Applying the proposed method in the Least-Squares Policy Iteration (LSPI) framework, we observe superior performance compared to widely used parametric basis functions on two standard benchmarks in terms of policy quality.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Ilya Kostrikov
- Denis Yarats
- Sergey Levine
- Rob Fergus
- Pieter Abbeel
- Abhinav Kumar Gupta
- Ruslan Salakhutdinov
- Yoshua Bengio
Citations in the corpus
- Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels (2020, methodology): “Regularization in RL Some early attempts to learn RL function approximators used (cid:96) 2 regularization of the Q [16, 48] function.”