Modeling Interaction via the Principle of Maximum Causal Entropy

By Brian D. Ziebart, J. Bagnell, Anind K. Dey. 2010

393 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Molecular Biology, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Molecular Biology

In Reinforcement Learning

Citations in the corpus

Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review (2018, background): “For this reason, Ziebart and colleagues proposed the maximum causal entropy framework for inverse reinforcement learning under stochastic dynamics (Ziebart et al., 2010).”
Generative Adversarial Imitation Learning (2016, methodology): “[30], simplifies our discussion and in fact guarantees the existence of c̃ ∈ IRLψ(πE).”
A unified view of entropy-regularized Markov decision processes (2017, background): “, 1997; Ruszczyński, 2010), or to model observed behavior of imperfect decision-makers (Ziebart et al., 2010; Ziebart, 2010; Braun et al., 2011), among others.”
A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress (2018, background): “[46] show that the robustness of maxent against an imperfect demonstration is better than that of mmp.”
Multi-task Maximum Entropy Inverse Reinforcement Learning (2018, background): “By contrast, maximum causal entropy chooses a single reward function, using the principle of maximum entropy to select the least speciﬁc reward function that is still consistent with the demonstrations (Ziebart et al., 2008; 2010).”
Theory of Mind and Preference Learning at the Interface of Cognitive Science, Neuroscience, and AI: A Review (2022, background): “Further, Ziebart et al. (2010) replace the maximum entropy in the previous algorithm with the maximum causal entropy, allowing for a more principled algorithm in which each action’s entropy is only conditioned on the previous states, and not future states.”
CHOMP: Covariant Hamiltonian optimization for motion planning (2013, background): “Beyond the potential benefits to optimization of a path sampling procedure, the ability to model distributions over the space of paths is a problem of fundamental significance to statistical physics [10], finance [37], modeling the behavior of people [71, 72], and recent models of robust and stochas”
SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards (2019, methodology): “SQIL is built on soft Q-learning, which assumes that expert behavior follows the maximum entropy model (Ziebart et al., 2010; Levine, 2018).”
XIRL: Cross-embodiment Inverse Reinforcement Learning (2021, methodology): “Inspired by maximum entropy inverse RL [24, 25] and generative adversarial networks [26], the seminal GAIL algorithm [27] performs distribution matching between the expert and policy’s state-action occupancy via an adversarial formulation; a discriminator is trained with on-policy samples, which is ”
An Invitation to Imitation (2015, methodology): “, 2010, 2013] Moreover, some methods, like that of [Ziebart et al., 2010], have yet another interpretation in terms of optimal control perturbed by certain shocks that are not visible to the learner.”
Reward is enough (2021, background): “The reinforcement learning problem may also be transformed into a probabilistic framework that approximates the goal of reward maximisation [66,39,26,17].”
Modeling Purposeful Adaptive Behavior with the Principle of Maximum Causal Entropy (2010, background): “• Maximum causal entropy influence diagrams (Ziebart et al., 2010b) expand the maximum causal entropy approach to settings with addition uncertainty over side information, such as learning to model diagnostic decision making.”
Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning (2018, methodology): “Motivated by the task of decision prediction in sequential interactions, Ziebart, Bagnell, and Dey (2010) propose to use maximum causal entropy to model the availability and inﬂuence of sequentially revealed side information.”
Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap (2021, methodology): “A second option is to optimize over the causal polytope P (A ||S ), as defined in (Ziebart et al. 2010).”
Relative Entropy Inverse Reinforcement Learning (2011, methodology): “In this paper, we build on the Maximum Entropy framework (Ziebart et al., 2008, 2010) and introduce a novel model-free algorithm for Inverse Reinforcement Learning.”
Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamics (2016, methodology): “…of a speciﬁc trajectory τ based on the start distribution, the single actions and the transitions: Ziebart et al. have shown in (Ziebart, 2010; Ziebart et al., 2010) that appropriate feature weights can be learned by optimizing the likelihood of the data under the maximum causal entropy…”