Nonlinear Inverse Reinforcement Learning with Gaussian Processes

By Sergey Levine, Zoran Popovic, Vladlen Koltun. 2011

410 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

One-Shot Imitation Learning (2017, background): “…Pomerleau (1989); Ross et al. (2011)); and inverse reinforcement learning (Ng & Russell, 2000), where a reward function (Abbeel & Ng, 2004; Ziebart et al., 2008; Levine et al., 2011; Finn et al., 2016; Ho & Ermon, 2016) is estimated that explains the demonstrations as (near) optimal behavior.”
Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review (2018, background): “…φ. Subsequent work in inverse reinforcement learning has studied settings where the reward function has a more complex, non-linear representation (Levine et al., 2011; Wulfmeier et al., 2015), and extensions to approximate inference via the Laplace approximation (under known dynamics) (Levine…”
Learning Parameterized Skills (2012, methodology): “This optimization problem is related to that of Inverse Reinforcement Learning (IRL), where the goal is to identify a reward function under which an optimal policy matches sample trajectories provided by an expert [52, 43].”
Value Iteration Networks (2016, methodology): “A similar approach is also used in inverse reinforcement learning (IRL) (Abbeel & Ng, 2004; Ziebart et al., 2008; Levine et al., 2011), which has been used for similar domains of predicting trajectories from overhead images as in this work (Kitani et al.”
Generative Adversarial Imitation Learning (2016, background): “Using expressive cost function classes, like Gaussian processes [14] and neural networks [6], is crucial to properly explain complex expert behavior without meticulously hand-crafted features.”
Reinforcement Learning in Healthcare: A Survey (2019, methodology): “few years. Numerous IRL methods have been proposed, including the Maximum Entropy IRL [55], the Apprenticeship Learning [56], nonlinear representations of the reward function using Gaussian processes [57], and Bayesian IRL [58]. c) Representation for Tasks or Models: Much recent research on RL has f”
Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation (2018, background): “iseases and complex medication space. Methods of combing SL and RL utilize expert behaviors to accelerate reinforcement learning and avoid the risks of actions. Common examples are imitation learning [1, 12, 19, 28, 44] and supervised actor-critic [5, 6, 10]. Given the samples of expert trajectories”
Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation (2017, background): “…within imitation learning are behavioral cloning, which performs supervised learning from observations to actions (e.g., [4], [24]) and inverse reinforcement learning [25], where a reward function [26], [27], [28], [29], [30] is estimated to explain the demonstrations as (near) optimal behavior.”
Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization (2016, methodology): “, 2013) and some have proposed using nonlinear cost functions (Ratliff et al., 2006; 2009; Levine et al., 2011), to our knowledge no prior method has been demonstrated that can provide all of these benefits in the context of complex real-world tasks.”
Third-Person Imitation Learning (2017, background): “Recently, there has been a signiﬁcant amount of work on IRL, and current algorithms can infer a reward function from a very modest number of demonstrations (e.g,. Abbeel & Ng (2004); Ratliff et al. (2006); Ziebart et al. (2008); Levine et al. (2011); Ho & Ermon (2016); Finn et al. (2016)).”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “To learn a nonlinear reward function, Levine et al. [2011] use a Gaussian Process (GP) approach based on the maximum entropy principle [Ziebart et al., 2008]. The original maximum entropy based approach Ziebart et al. [2008] uses linear reward features for the reward function. Levine et al. [2011] u”
Learning Robust Rewards with Adversarial Inverse Reinforcement Learning (2017, methodology): “Previous methods which aim to learn nonlinear cost functions have used boosting (Ratliff et al., 2007) and Gaussian processes (Levine et al., 2011), but still suffer from the feature engineering problem.”
Concrete Problems for Autonomous Vehicle Safety: Advantages of Bayesian Deep Learning (2017, methodology): “Variants of IRL include Bayesian IRL which use Gaussian processes [Levine et al., 2011], as do deep IRL which use neural networks [Wulfmeier et al., 2015a].”
Maximum Entropy Deep Inverse Reinforcement Learning (2015, methodology): “We assess the performance of DeepIRL two benchmark tasks against current state-of-the-art approaches : GPIRL (Levine et al., 2011), NPB-FIRL (Choi & Kim, 2013) and the original MaxEnt (Ziebart et al., 2008) to illustrate the necessity of non-linear function approximation.”
Model-Free Imitation Learning with Policy Optimization (2016, background): “Nonlinear cost function classes have been successful in IRL (Ratliff et al., 2009; Levine et al., 2011) as well as in other machine learning problems reminiscent of ours, in particular that of training generative image models.”
Prediction of Human Activity by Discovering Temporal Sequence Patterns (2014, background): “However, for activity prediction, it tries to infer the intention or a higher level activity class with observation of only a few action units.”
Continuous Inverse Optimal Control with Locally Optimal Examples (2012, methodology): “In the nonlinear variant of our algorithm, we represent the reward function as a Gaussian process (GP) that maps from feature values to rewards, as proposed by Levine et al. (Levine et al., 2011).”
Inverse reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units (2019, methodology): “Numerous IRL methods have been proposed afterwards, including Apprenticeship Learning [11], Maximum Entropy IRL [23], Bayesian IRL [24], and nonlinear representations of the reward function using Gaussian processes [25].”
Hybrid Reinforcement Learning with Expert State Sequences (2019, methodology): “On the other hand, IRL methods aim to infer the goal of an expert, expressed as a reward function (Ng and Russell 2000; Abbeel and Ng 2004; Ziebart et al. 2008; Levine, Popovic, and Koltun 2011; Borsa et al. 2017).”
Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Nets (2017, background): “The majority of IRL works [16, 33, 1, 12, 19] introduce algorithms that can imitate a single skill from demonstrations thereof but they do not readily generalize to learning a multi-task policy from a set of unstructured demonstrations of various tasks.”