Latent Kullback Leibler Control for Continuous-State Systems using Probabilistic Graphical Models

By Takamitsu Matsubara, Vicenç Gómez, Hilbert J. Kappen. 2014

22 citations recorded by the source corpus.

Kullback Leibler (KL) control problems allow for efficient computation of optimal control by solving a principal eigenvector problem. However, direct applicability of such framework to continuous state-action systems is limited. In this paper, we propose to embed a KL control problem in a probabilistic graphical model where observed variables correspond to the continuous (possibly high-dimensional) state of the system and latent variables correspond to a discrete (low-dimensional) representation of the state amenable for KL control computation. We present two examples of this approach. The first one uses standard hidden Markov models (HMMs) and computes exact optimal control, but is only applicable to low-dimensional systems. The second one uses factorial HMMs, it is scalable to higher dimensional problems, but control computation is approximate. We illustrate both examples in several robot motor control tasks.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Control and Systems Engineering, Optimization & Training, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dit-Yan Yeung
2. Hao Wang
3. Yu Zhang
4. Qiang Yang
5. Xing Xie
6. Yu Zhang
7. Mingsheng Long
8. Jianmin Wang

In Computational Theory and Mathematics

1. Dit-Yan Yeung
2. Hao Wang

In Control and Systems Engineering

1. Dit-Yan Yeung
2. Hao Wang

In Optimization & Training

1. Dit-Yan Yeung
2. Hao Wang
3. Qiang Yang
4. Yu Zhang
5. Yu Zhang

In Probabilistic & Bayesian Methods

1. Dit-Yan Yeung
2. Hao Wang

Citations in the corpus

• Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images (2015, neutral): “, hidden Markov models and Kalman filters[33, 34, 35].”
• Towards Bayesian Deep Learning: A Survey (2016, methodology): “The perception task can be taken care of using multiple layers of simple nonlinear transformation (deep learning) while the control task usually needs more sophisticated models like hidden Markov models and Kalman filters [21], [38].”
• A Survey on Bayesian Deep Learning (2016, methodology): “The perception task of processing raw images can be handled by deep learning while the control task usually needs more sophisticated models such as hidden Markov models and Kalman filters [35, 74].”