Algorithms for Inverse Reinforcement Learning

By Andrew Y. Ng, Stuart J. Russell. 2000

2,519 citations recorded by the source corpus.

Subfields: Infectious Diseases, Small Animals, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Infectious Diseases

In Small Animals

In Reinforcement Learning

Citations in the corpus

Deep Reinforcement Learning from Human Preferences (2017, methodology): “If we have demonstrations of the desired task, we can extract a reward function using inverse reinforcement learning (Ng and Russell, 2000).”
One-Shot Imitation Learning (2017, background): “The former can be time-consuming and the latter can often be significantly more difficult than providing a demonstration (Ng & Russell, 2000).”
Better Rewards Yield Better Summaries: Learning to Summarise Without References (2019, background): “However, ROUGE performs poorly at summary level : given multiple summaries for the same input document, ROUGE can hardly distinguish the “good” summaries from the “mediocre” and “bad” ones (Novikova et al., 2017).”
Scalable agent alignment via reward modeling: a research direction (2018, background): “IRL is an under-constrained problem because the reward function is not uniquely identiﬁ-able (not even up to afﬁne-linear transformation) from behavior alone (Ng & Russell, 2000); for example, R = 0 is always a solution.”
Reward learning from human preferences and demonstrations in Atari (2018, methodology): “This is closely related to generative adversarial imitation learning (Ho and Ermon, 2016), a form of IRL.”
Supervising strong learners by amplifying weak experts (2018, background): “The bottom center box (RL from a human training signal) is beginning to be explored, and includes inverse reinforcement learning (Ng and Russell, 2000; Abbeel and Ng, 2004; Finn et al., 2016) and RL from human feedback (Knox and Stone, 2009; Pilarski et al., 2011; MacGlashan et al., 2017;…”
Learning to Understand Goal Specifications by Modelling Reward (2018, background): “When there is a single implicit task, Inverse Reinforcement Learning (IRL; Ng & Russell, 2000; Ziebart et al., 2008) methods in general, and Generative Adversarial Imitation Learning (Ho & Ermon, 2016) in particular, have yielded some success in jointly learning reward functions from expert data…”
Learning Dynamic Robot-to-Human Object Handover from Human Feedback (2016, background): “One approach is inverse reinforcement learning (IRL), also called inverse optimal control, which learns a reward function from expert demonstration [22, 24].”
Pre-Trained Language Models for Interactive Decision-Making (2022, methodology): “All our experiments use imitation learning (Santara et al., 2017; Ng et al., 2000; Peng et al., 2018), specifically”
Shaking the foundations: delusions in sequence models for interaction and control (2021, background): “Remark 13.”
Learning Parameterized Skills (2012, methodology): “This optimization problem is related to that of Inverse Reinforcement Learning (IRL), where the goal is to identify a reward function under which an optimal policy matches sample trajectories provided by an expert [52, 43].”
Generative Adversarial Imitation Learning (2016, background): “There are two main approaches suitable for this setting: behavioral cloning [18], which learns a policy as a supervised learning problem over state-action pairs from expert trajectories; and inverse reinforcement learning [23, 16], which finds a cost function under which the expert is uniquely optim”
Aligning AI With Shared Human Values (2020, background): “Others take a more empirical approach and use inverse reinforcement learning (Ng and Russell, 2000) to learn task-specific individual preferences about trajectories from scratch (Christiano et al., 2017).”
Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, background): “IRL is the problem of extracting a reward function given observed, optimal behavior [72].”
Reinforcement Learning in Healthcare: A Survey (2019, methodology): “s sometimes preferable to directly estimate the reward function of experts from a set of presumably optimal treatment trajectories in retrospective medical data. Imitation learning, particularly, IRL [53], [54], is one of the most feasible approaches to infer reward functions given observations of o”
SHOPPER: A Probabilistic Model of Consumer Choice with Substitutes and Complements (2017, background): “From the machine learning perspective, it closely resembles themotivation behind inverse reinforcement learning (Russell, 1998; Ng and Russell, 2000).”
Stochastic Primal-Dual Q-Learning (2018, neutral): “…as the distributed RL (Kar et al., 2013; Macua et al., 2015; Zhang et al., 2018; Lee et al., 2018), Q-learning with function approximations (Sutton et al., 2009b,a), sparsity promoted RL (Mahadevan and Liu, 2012), safe RL (Garcıa and Fern´andez, 2015), and the inverse RL (Ng and Russell, 2000).”