Robot trajectory optimization using approximate inference

By Marc Toussaint. 2009

463 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Molecular Biology, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Molecular Biology

In Reinforcement Learning

Citations in the corpus

Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review (2018, background): “…inference and control has been explored in the literature under a variety of names, including the Kalman duality (Todorov, 2008), maximum entropy reinforcement learning (Ziebart, 2010), KLdivergence control (Kappen et al., 2012; Kappen, 2011), and stochastic optimal control (Toussaint, 2009).”
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018, methodology): “This framework has been used in many contexts, from inverse reinforcement learning (Ziebart et al., 2008) to optimal control (Todorov, 2008; Toussaint, 2009; Rawlik et al., 2012).”
Reinforcement Learning with Deep Energy-Based Policies (2017, background): “The solution can be shown to optimize an entropy-augmented reinforcement learning objective or to correspond to the solution to a maximum entropy learning problem (Toussaint, 2009).”
Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images (2015, methodology): “As an upper bound on the performance achievable for control by any of the models we also computed the true system cost by applying iLQR/AICO to a model of the real system dynamics.”
Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search (2018, methodology): “This policy improvement step can be motivated by the framework of RL as variational inference (Toussaint, 2009) and is equivalent to minimizing the KL divergence to a trajectory distribution ∝ exp(G)pπk which puts additional mass on high-return trajectories.”
Maximum a Posteriori Policy Optimisation (2018, methodology): “Intuitively, O can be interpreted as the event of obtaining maximum reward by choosing an action ; or the event of succeeding at the RL task (Toussaint, 2009; Neumann, 2011).”
Reinforcement Learning from Imperfect Demonstrations (2018, methodology): “Our approach, Normalized Actor-Critic (NAC), uses a uniﬁed loss function to process both off-line demonstration data and on-line experience based on the underlying maximum entropy reinforcement learning framework (Toussaint, 2009; Haarnoja et al., 2017b; Schulman et al., 2017).”
Soft Actor-Critic Algorithms and Applications (2018, methodology): “, 2018c), we introduced the Soft Actor-Critic (SAC) algorithm based on the maximum entropy framework (Ziebart et al., 2008; Toussaint, 2009; Rawlik et al., 2012; Fox et al., 2016; Haarnoja et al., 2017).”
A Divergence Minimization Perspective on Imitation Learning Methods (2019, background): “The connections between RL and divergence minimization have long been studied in the rich prior literature of control as probabilistic inference [17, 18, 19, 20, 21, 22, 23].”
Latent Space Policies for Hierarchical Reinforcement Learning (2018, methodology): “Our approach is based on a formulation of reinforcement learning as inference in a graphical model (Ziebart et al., 2008; Toussaint, 2009; Levine, 2014).”
Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control (2017, methodology): “For a linear fMM and a quadratic cost ` we can write the LQG case as shown in Theorem 1 in [47].”
Generating Legible Motion (2013, background): “IV), echoing earlier works in motion planning [9, 21, 22, 27, 29, 33, 35], now with legibility as an optimization criterion.”
The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models (2025, unknown): “Entropy-regularized reinforcement learning, also referred as maximum entropy RL (Ziebart et al., 2008; Toussaint, 2009), adopts a regularization term in reward to encourage high-entropy actions.”
Combining the benefits of function approximation and trajectory optimization (2014, methodology): “Here we use the Alternating Direction Method of Multipliers (ADMM) [5] to couple the network and trajectory optimizer, while in [13] the coupling was done by casting the control problem as a dual Bayesian inference problem [37] and exploiting variational approximations.”
How to train your robot with deep reinforcement learning: lessons we have learned (2021, methodology): “Numerous methods for improving exploration have been proposed in the literature (Ziebart et al. 2008; Toussaint 2009; Rawlik et al. 2013; Fox et al. 2016; Osband et al. 2016; Pathak et al. 2017; Haarnoja et al. 2017), and many of these can be applied directly to real-world robotic RL.”
Learning to Walk via Deep Reinforcement Learning (2018, methodology): “This framework has been used in many contexts, from inverse reinforcement learning [46] to optimal control [43, 44, 36].”
Spot and Learn: A Maximum-Entropy Patch Sampler for Few-Shot Image Classification (2019, methodology): “Our model aims to produce possible patch sequences from an input that would lead to correct classification by applying the maximum entropy reinforcement learning objective [26].”
Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics (2014, methodology): “Time-varying linear-Gaussian policies have previously been used in a number of model-based and model-free methods [25, 16, 14] due to their close connection with linear feedback controllers, which are frequently used in classic deterministic trajectory optimization.”