Inverse Optimal Control with Linearly-Solvable MDPs

By Krishnamurthy (Dj) Dvijotham, E. Todorov. 2010

170 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review (2018, methodology): “In this setup, the algorithm learns the advantage function directly, rather than the reward, which corresponds roughly to an adversarial version of the OptV algorithm (Dvijotham and Todorov, 2010).”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “Dvijotham and Todorov [2010] finds the maximum likelihood value function from an unconstrained convex optimization problem.”
A Cascaded Supervised Learning Approach to Inverse Reinforcement Learning (2013, methodology): “The algorithm of [3] also advertises not having to solve the RL problem, but needs to deal with linearly solvable MDPs, therefore we do not include it in our tests.”
Generalizing Skills with Semi-Supervised Reinforcement Learning (2016, methodology): “In order to perform semi-supervised reinforcement learning, we use the framework of maximum entropy control (Ziebart, 2010; Kappen et al., 2012), also called linear-solvable MDPs (Dvijotham & Todorov, 2010).”
Multi-robot inverse reinforcement learning under occlusion with interactions (2014, background): “[9] K. Dvijotham and E. Todorov.”
Hierarchy Through Composition with Multitask LMDPs (2017, background): “Linearly Solvable MDPs The LMDP (Todorov, 2009a; Dvijotham & Todorov, 2010; Todorov, 2009b; Dvijotham & Todorov, 2010) is defined by a three-tuple L = 〈S, P,R〉, where S is a set of states, P is a passive transition probability distribution P : S × S → [0, 1], and R is an expected instantaneous…”
Concept Learning with Energy-Based Models (2018, methodology): “Max entropy [36] is a common IRL formulation, and our method closely resembles recent instantiations of it [8, 5].”
Recent Advances in Robot Learning from Demonstration (2020, methodology): “In the continuous case, the methods become very akin to inverse optimal control (IOC) where a hidden objective function for optimal control is estimated from demonstrations (114, 115, 116, 117), and both terms are often used equivalently in the literature.”
Guided Policy Search (2013, methodology): “We now show how the same algorithm can be used with the framework of linearly solvable MDPs (Dvijotham & Todorov, 2010) and the related concept of maximum entropy control (Ziebart, 2010) to build an approximate Gaussian I-projection of ρ(ζ) ∝ exp(r(ζ)).”
Approximate MaxEnt Inverse Optimal Control and Its Application for Mental Simulation of Human Interactions (2015, background): “2008), though one may estimate the value function directly (Dvijotham and Todorov 2010) or solve a single large quadratic program (Ratliff, Bagnell, and Zinkevich 2006).”
Nonlinear Inverse Reinforcement Learning with Gaussian Processes (2011, methodology): “This model is convenient for IRL, because its likelihood is differentiable [17], and a complete stochastic policy uniquely determines the reward function [3].”
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms (2019, background): “Finally, in the more restricted case of linearly-solvable MDPs, the IRL problem is well-posed, avoiding these problems (Dvijotham and Todorov, 2010).”
Continuous Inverse Optimal Control with Locally Optimal Examples (2012, background): “We can ﬁnd a suitable weight simply by doubling θ r until this requirement is met.”
Learning objective functions for manipulation (2013, methodology): “[11] proposes an efficient solution which uses samples to approximate the value function directly based on a linear MDP formulation.”