Bayesian Inverse Reinforcement Learning

By Deepak Ramachandran, Eyal Amir. 2007

882 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Reinforcement Learning in Healthcare: A Survey (2019, methodology): “To address this problem, Yu et al. [250] applied IRL methods in sepsis treatment to infer the reward functions of clinicians during their treatment process.”
Machine Theory of Mind (2018, methodology): “Rather than using the computationally costly algorithms of classical inverse reinforcement learning (e.g. Ng et al., 2000; Ramachandran & Amir, 2007; Ziebart et al., 2008; Boularias et al., 2011), or Bayesian ToM (e.g. Baker et al., 2011; Nakahashi et al., 2016; Baker et al., 2017), we drive the…”
Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization (2016, methodology): “Prior work has tackled this issue using maximum margin formulations (Abbeel & Ng, 2004; Ratliff et al., 2006), as well as probabilistic models that explain suboptimal behavior as noise (Ramachandran & Amir, 2007; Ziebart et al., 2008).”
Cooperative Inverse Reinforcement Learning (2016, methodology): “Ng & Russell (2000), Abbeel & Ng (2004), and Ratliff et al. (2006) compute constraints that characterize the set of reward functions so that the observed behavior maximizes reward. In general, there will be many reward functions consistent with this constraint. They use a max-margin heuristic to sel”
Third-Person Imitation Learning (2017, background): “…to a trajectory (Calinon et al., 2007; Abbeel et al., 2010), as a weighted combination of features (Abbeel & Ng, 2004; Ratliff et al., 2006; Ramachandran & Amir, 2007; Ziebart et al., 2008; Boularias et al., 2011; Kalakrishnan et al., 2013; Doerr et al., 2015), or could also involve…”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “Ramachandran and Amir [2007] proposed an IRL method based on the Bayesian framework.”
Unsupervised Perceptual Rewards for Imitation Learning (2016, methodology): “Inverse reinforcement learning can be performed with a variety of algorithms (Ng et al., 2000), ranging from margin-based methods (Abbeel & Ng, 2004; Ratliff et al., 2006) to methods based on probabilistic models (Ramachandran & Amir, 2007; Ziebart et al., 2008).”
Maximum Entropy Deep Inverse Reinforcement Learning (2015, methodology): “This may be done either to learn the reward structure for modelling purposes or to provide a method to allow the agent to imitate a demonstrator’s specific behaviour (Ramachandran and Amir 2007).”
Inverse Reinforcement Learning in Swarm Systems (2016, methodology): “cy.1 In the subsequent sections, we show that this symmetry property also translates to the value functions of the agents. Algorithmically, we exploit the fact that most existing IRL methods, such as [2, 19, 20, 24, 29, 35], share a common generic form (Algorithm 1), which involves three main steps ”
Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations (2019, background): “However, existing IRL algorithms fail to do this, typically searching for a reward function that makes the demonstrations appear near-optimal (Ramachandran & Amir, 2007; Ziebart et al., 2008; Finn et al., 2016; Henderson et al., 2018).”
Learning socially normative robot navigation behaviors with Bayesian inverse reinforcement learning (2016, methodology): “We build upon this work by adding improved exploration and then leverage the resulting flexible representation to also improve upon Bayesian IRL algorithm of Ramachandran and Amir [15].”
Learning from Limited Demonstrations (2013, background): “There are a few Inverse RL methods that do not assume optimal experts [25, 26], but their focus is on learning the reward function rather than on planning.”
Towards Resolving Unidentifiability in Inverse Reinforcement Learning (2016, background): “Since the true reward function is fundamentally unidentiﬁ-able, much of the previous work in IRL has been concerned with the development of heuristics which prefer certain re-wards as better explanations for behavior than others [10, 20, 11].”
Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamics (2016, methodology): “Many approaches have been proposed to solve the IRL problem, such as (Abbeel and Ng, 2004; Ratliff et al., 2006; Neu and Szepesvári, 2007; Ramachandran and Amir, 2007; Rothkopf and Dimitrakakis, 2011).”
A Computational Model of Social-Learning Mechanisms (2009, methodology): “3 The method uses MCMC to estimate the distribution over the space of possible u -functions, given the demonstration (Ramachandran & Amir, 2007).”
Apprenticeship Learning About Multiple Intentions (2011, background): “Bayesian IRL (Ramachandran & Amir, 2007) is a framework for estimating posterior probabilities over possible reward functions given the observed trajectories.”