Relative Entropy Inverse Reinforcement Learning

By Abdeslam Boularias, Jens Kober, Jan Peters. 2011

397 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Dueling RL: Reinforcement Learning with Trajectory Preferences (2021, background): “In Ng et al. (2000); Boularias et al. (2011); Neu and Szepesvári (2012); Wulfmeier et al. (2015), inverse reinforcement learning problems have been considered, where the objective is to extract (unknown) reward function from the trajectories given by an oracle or expert.”
Data-efficient Deep Reinforcement Learning for Dexterous Manipulation (2017, methodology): “Finally, the use of demonstration data has played an important role in robot learning, both as a means to obtain suitable cost functions [2, 14, 4, 8] but also to bootstrap and thus speed up learning.”
ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning through Imitation (2018, methodology): “Prior works show that data-driven IL methods improve both the sample efﬁciency of policy learning [16, 17, 18] and the performance of the trained policy [19, 20, 21].”
Machine Theory of Mind (2018, methodology): “Rather than using the computationally costly algorithms of classical inverse reinforcement learning (e.g. Ng et al., 2000; Ramachandran & Amir, 2007; Ziebart et al., 2008; Boularias et al., 2011), or Bayesian ToM (e.”
Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization (2016, methodology): “Prior sample-based IOC methods use a linear representation of the cost function, which simpliﬁes the corresponding cost learning problem (Boularias et al., 2011; Kalakrishnan et al., 2013).”
Reinforcement and Imitation Learning for Diverse Visuomotor Skills (2018, background): “Cost functions can be derived from demonstrations either via tracking objectives (e.g. Gupta et al. [11]) or via inverse RL (e.g. Boularias et al. [1], Finn et al. [6]), or, as in our case, via adversarial learning [15].”
Third-Person Imitation Learning (2017, background): “…et al., 2010), as a weighted combination of features (Abbeel & Ng, 2004; Ratliff et al., 2006; Ramachandran & Amir, 2007; Ziebart et al., 2008; Boularias et al., 2011; Kalakrishnan et al., 2013; Doerr et al., 2015), or could also involve feature learning (Ratliff et al., 2007; Levine et al.,…”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “It is often useful to consider the maximum entropy principle in its regularized form [Ziebart et al., 2013] [Boularias et al., 2011], that is, instead of finding a maximum entropy distribution we want to find a distribution with the minimal KL divergence relative to a “prior” distribution p0(τ )…”
A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models (2016, methodology): “Sample-based algorithms for performing maximum entropy (MaxEnt) IRL have scaled cost learning to scenarios with unknown dynamics, using nonlinear function classes, such as neural networks [4, 11, 7].”
Learning Robust Rewards with Adversarial Inverse Reinforcement Learning (2017, background): “Previous IRL approaches have included maximum margin approaches (Abbeel & Ng, 2004; Ratliff et al., 2006), and probabilistic approaches such as Ziebart et al. (2008); Boularias et al. (2011).”
Occam's razor is insufficient to infer the preferences of irrational agents (2017, background): “ot trivial for people. 1 Few IRL approaches have been tried to deal with these problems. Some deal with speciﬁc biases [Evans et al.,2015b,a], whereasothersassumenoisyrationality[Ziebart et al., 2008,Boularias et al., 2011]. A general framework for inferring the reward from suboptimal behavior does ”
Unsupervised Perceptual Rewards for Imitation Learning (2016, methodology): “Prior work has addressed this using dynamic programming to exactly compute Z in small, discrete systems (Ziebart et al., 2008), or by using sampling to estimate Z for large, continuous domains (Kalakrishnan et al., 2010; Boularias et al., 2011; Finn et al., 2016b).”
Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations (2019, background): “Maximum entropy IRL does this matching, while further disambiguating inference by also maximizing the entropy of the resulting policy (Ziebart et al., 2008; Boularias et al., 2011; Wulfmeier et al., 2015; Finn et al., 2016).”
Policies for Active Learning from Demonstration (2017, methodology): “A recent example of the application of IRL is the work of Boularias, Kober, and Peters (2011), where they focus on generalization from a small number of samples using an extension of maximum entropy IRL.”
A Function Approximation Method for Model-based High-Dimensional Inverse Reinforcement Learning (2017, methodology): “The methods in [13], [21] update the reward function parameter by minimizing the relative entropy between the observed trajectories and a set of sampled trajectories based on the reward function, but they require a set of manually segmented trajectories of human motion, where the choice of…”
A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges (2023, neutral): “combined into a reward, such as the distance between the ball and the cup for a robotic ball-in-cup game [60].”
An Introduction to Deep Reinforcement Learning (2018, positive): “Note that recent works bypass the requirement of the knowledge of the system dynamics (Boularias et al., 2011; Kalakrishnan et al., 2013; Finn et al., 2016b) by using techniques based on the principle of maximum causal entropy (Ziebart, 2010).”
Learning objective functions for manipulation (2013, methodology): “[12] uses a sampling-based approach combined with importance sampling to arrive at an objective function similar to the maximum entropy method [10].”
Learning strategies in table tennis using inverse reinforcement learning (2014, methodology): “Boularias et al. (2011) already used the same sample-based technique described in the previous paragraph.”