Apprenticeship learning via inverse reinforcement learning

By Pieter Abbeel, Andrew Y. Ng. 2004

3,716 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model (2019, methodology): “While the learned short-run MCMC can be used for synthesis, the above learning scheme can be generalized to tasks such as image inpainting, super-resolution, style transfer, or inverse optimal control [52, 1] etc.”
One-Shot Imitation Learning (2017, background): “…to actions (e.g., Pomerleau (1989); Ross et al. (2011)); and inverse reinforcement learning (Ng & Russell, 2000), where a reward function (Abbeel & Ng, 2004; Ziebart et al., 2008; Levine et al., 2011; Finn et al., 2016; Ho & Ermon, 2016) is estimated that explains the demonstrations as…”
Scalable agent alignment via reward modeling: a research direction (2018, background): “…interaction that have been explored in the literature: providing a set of demonstrations of the desired behavior (Russell, 1998; Ng & Russell, 2000; Abbeel & Ng, 2004; Argall et al., 2009); providing feedback in the form of scores (El Asri et al., 2016), actions (Grifﬁth et al., 2013), value (Knox…”
Reward learning from human preferences and demonstrations in Atari (2018, background): “This is closely related to generative adversarial imitation learning (Ho and Ermon, 2016), a form of IRL.”
Supervising strong learners by amplifying weak experts (2018, background): “…bottom center box (RL from a human training signal) is beginning to be explored, and includes inverse reinforcement learning (Ng and Russell, 2000; Abbeel and Ng, 2004; Finn et al., 2016) and RL from human feedback (Knox and Stone, 2009; Pilarski et al., 2011; MacGlashan et al., 2017; Christiano…”
Learning to Understand Goal Specifications by Modelling Reward (2018, background): “Many approaches to apprenticeship learning are variants of inverse reinforcement learning (IRL), which aims to recover a reward function from expert demonstrations (Abbeel & Ng, 2004; Ziebart et al., 2008).”
Shaking the foundations: delusions in sequence models for interaction and control (2021, background): “This is a simple question with far-reaching implications on foundation models [4], language agents [5], behavior cloning [10], inverse reinforcement learning [1, 13], goal-/return-conditioned policies [6, 11, 22]), and so forth.”
Bridging the Gap Between Value and Policy Based Reinforcement Learning (2017, methodology): “Incorporating expert trajectories in PCL is relatively trivial compared to the specialized methods developed for other policy based algorithms [1, 12].”
Value Iteration Networks (2016, methodology): “We demonstrate that VINs can be effectively used within standard RL and IL algorithms in various problems, among which require visual perception, continuous control, and also natural language based decision making in the WebNav challenge [20].”
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning (2010, methodology): “strations of good behavior are used to learn a controller, have proven very useful in practice and have led to stateof-the art performance in a variety of applications (Schaal, 1999; Abbeel and Ng, 2004; Ratliff et al., 2006; Silver et al., 2008; Argall et al., 2009; Chernova and Veloso, 2009; Ross ”
Generative Adversarial Imitation Learning (2016, methodology): “[10] using the cost function class Clinear (10) of Abbeel and Ng [1]”
Time-Contrastive Networks: Self-Supervised Learning from Video (2017, background): “to learn a reward function that can be used to optimize an imitation policy with reinforcement learning [10].”
Learning World Graphs to Accelerate Hierarchical Reinforcement Learning (2019, neutral): “Drawing intuition from unsupervised sequence segmentation [17, 47] and imitation learning [1, 46], we define Vp as the most critical states in recovering action sequences generated by some agent, indicating that these states lead to the most information gain [4].”
Plan explicability and predictability for robot task planning (2015, background): “In the human-robot interaction (HRI) community, there exists prior works that discuss how to enable natural and fluent human-robot interaction [11, 12, 21, 18] to create more socially acceptable robots [7].”
Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation (2020, background): “Leveraging human guidance for RL tasks has been extensively studied in the context of imitation learning (learning from demonstration) [37], inverse reinforcement learning [29, 1], reward shaping [28], learning from human preference [7, 22], and learning from saliency information provided by human [”
Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, methodology): “To avoid degenerating a solution which would ﬁt the reward but not the original behaviour, authors [71] proposed a method for enforcing that the optimal policy learnt over the rewards should still match the observed policy in behavior.”
Reinforcement Learning in Healthcare: A Survey (2019, methodology): “Numerous IRL methods have been proposed, including the Maximum Entropy IRL [55], the Apprenticeship Learning [56], nonlinear representations of the reward function using Gaussian processes [57], and Bayesian IRL [58].”
Verifiable Reinforcement Learning via Policy Extraction (2018, methodology): “Our approach to policy extraction is based on imitation learning [27, 1], in particular, D AGGER [25]— the pretrained DNN policy (which we call the oracle ) is used to generate labeled data, and then supervised learning is used to train a decision tree policy.”
Deep Reinforcement Learning framework for Autonomous Driving (2017, methodology): “This is referred to as Inverse Reinforcement learning [1].”