Inferring The Latent Structure of Human Decision-Making from Raw Visual Inputs

By Yunzhu Li, Jiaming Song, Stefano Ermon. 2017

44 citations recorded by the source corpus.

The goal of imitation learning is to match example expert behavior, without access to a reinforcement signal. Expert demonstrations provided by humans, however, often show significant variability due to latent factors that are not explicitly modeled. We introduce an extension to the Generative Adversarial Imitation Learning method that can infer the latent structure of human decision-making in an unsupervised way. Our method can not only imitate complex behaviors, but also learn interpretable and meaningful representations. We demonstrate that the approach is applicable to high-dimensional environments including raw visual inputs. In the highway driving domain, we show that a model learned from demonstrations is able to both produce different driving styles and accurately anticipate human actions. Our method surpasses various baselines in terms of performance and functionality.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, positive): “In the ﬁeld of navigation, Li et al. (2017) successfully apply GAIL in simulated autonomous vehicle navigation scenarios with raw visual input.”
Visual Semantic Planning Using Deep Successor Representations (2017, background): “Recent works have employed a series of new techniques for imitation learning, such as generative adversarial networks [19, 29], Monte Carlo tree search [17] and guided policy search [27], which learn end-to-end policies from pixels to actions.”
InfoVAE: Information Maximizing Variational Autoencoders (2017, background): “mance metrics. 1. Introduction Generative models have shown great promise in modeling complex distributions such as natural images and text (Radford et al., 2015; Zhu et al., 2017; Yang et al., 2017; Li et al., 2017). These are directed graphical models which represent the joint distribution between”
InfoVAE: Balancing Learning and Inference in Variational Autoencoders (2019, background): “Generative models have shown great promise in modeling complex distributions such as natural images and text (Radford et al., 2015; Zhu et al., 2017; Yang et al., 2017; Li et al., 2017).”
Universal Planning Networks (2018, methodology): “This usage is inspired from the Deep Q Networks paper of Mnih et al. (2015) and similar metrics have also been used by Levine et al.”
One-Shot High-Fidelity Imitation: Training Large-Scale Deep Nets with RL (2018, unknown): “GAIL has been successfully applied in a number of different environments (Ho & Ermon, 2016; Li et al., 2017a; Merel et al., 2017; Zhu et al., 2018).”
Learning Policy Representations in Multiagent Systems (2018, background): “In another recent work, Li et al. (2017) propose an algorithm for learning interpretable representations using generative adversarial imitation learning.”
Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation (2017, methodology): “like algorithm called generative adversarial imitation learning (GAIL) [8], using a convolutional model to process images as suggested by [39].”
Socially Compliant Navigation Through Raw Depth Inputs with Generative Adversarial Imitation Learning (2017, methodology): “In line with [14], we extend the GAIL framework with Wasserstein GAN (WGAN) [16].”
A Lagrangian Perspective on Latent Variable Generative Models (2018, background): “Finally, elements in T3 are over both x and z, and they are empirically shown to be even more difficult to optimize (Li et al., 2017a).”
Learning a Prior over Intent via Meta-Inverse Reinforcement Learning (2018, methodology): “Several other methods have also proposed imitation learning approaches based on adversarial frameworks that resemble IRL, but do not aim to directly recover a reward function (Ho & Ermon, 2016; Li et al., 2017; Hausman et al., 2017; Kueﬂer & Kochenderfer, 2018).”