Value function approximation via low-rank models

By Hao Yi Ong. 2015

21 citations recorded by the source corpus.

We propose a novel value function approximation technique for Markov decision processes. We consider the problem of compactly representing the state-action value function using a low-rank and sparse matrix model. The problem is to decompose a matrix that encodes the true value function into low-rank and sparse components, and we achieve this using Robust Principal Component Analysis (PCA). Under minimal assumptions, this Robust PCA problem can be solved exactly via the Principal Component Pursuit convex optimization problem. We experiment the procedure on several examples and demonstrate that our method yields approximations essentially identical to the true function.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Weinan Zhang
2. Jun Wang

In Computer Vision and Pattern Recognition

1. Weinan Zhang
2. Jun Wang
3. M. Zhou
4. Yong Chen
5. Han Liu
6. Dell Zhang
7. Yaodong Yang
8. Ying Wen

Citations in the corpus

• Factorized Q-learning for large-scale multi-agent systems (2018, background): “In the reinforcement learning literature, the low-dimensionality (low-rank) assumption is widely regarded valid for single-agent, and researchers have utilized various approximation schemes to compactly represent the Q-function, e.g., Radial Basis Function (RBF), Cerebellar Model Articulation Contro”