Model-based imitation learning by probabilistic trajectory matching

By Péter Englert, Alexandros Paraschos, Jan Peters, Marc Peter Deisenroth. 2013

61 citations recorded by the source corpus.

One of the most elegant ways of teaching new skills to robots is to provide demonstrations of a task and let the robot imitate this behavior. Such imitation learning is a non-trivial task: Different anatomies of robot and teacher, and reduced robustness towards changes in the control task are two major difficulties in imitation learning. We present an imitation-learning approach to efficiently learn a task from expert demonstrations. Instead of finding policies indirectly, either via state-action mappings (behavioral cloning), or cost function learning (inverse reinforcement learning), our goal is to find policies directly such that predicted trajectories match observed ones. To achieve this aim, we model the trajectory of the teacher and the predicted robot trajectory by means of probability distributions. We match these distributions by minimizing their Kullback-Leibler divergence. In this paper, we propose to learn probabilistic forward models to compute a probability distribution over trajectories. We compare our approach to model-based reinforcement learning methods with hand-crafted cost functions. Finally, we evaluate our method with experiments on a real compliant robot.

Subfields: Artificial Intelligence, Biomedical Engineering, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Control and Systems Engineering

Citations in the corpus

End-to-End Driving Via Conditional Imitation Learning (2017, background): “Imitation learning has been applied to a variety of tasks, including articulated motion [2], [28], [11], autonomous flight [1], [13], [30], modeling navigational behavior [37], [38], off-road driving [22], [32], and road following [4], [6], [27], [36].”
Probabilistic model-based imitation learning (2013, background): “Matching the distribution of the predicted state trajectory for a given policy with the distribution over expert trajectories by means of minimizing the KL divergence induces a natural cost function in a standard RL context [16]: Equation (20) shows that matching two factorized distributions by mean”
Federated Imitation Learning: A Privacy Considered Imitation Learning Framework for Cloud Robotic Systems with Heterogeneous Sensor Data (2019, background): “At present, imitation learning has been applied to a variety of tasks, including autonomous flight [6], articulated motion [7], and end-to-end driving [8, 9].”
Model-based contextual policy search for data-efficient generalization of robot skills (2017, methodology): “While in [1], a time dependent forward model is used, we typically learn the dynamic model of the real hardware [34, 4, 29, 10, 18, 14].”