Model Predictive Path Integral Control using Covariance Variable Importance Sampling

By Grady Williams, Andrew Aldrich, Evangelos A. Theodorou. 2015

222 citations recorded by the source corpus.

In this paper we develop a Model Predictive Path Integral (MPPI) control algorithm based on a generalized importance sampling scheme and perform parallel optimization via sampling using a Graphics Processing Unit (GPU). The proposed generalized importance sampling scheme allows for changes in the drift and diffusion terms of stochastic diffusion processes and plays a significant role in the performance of the model predictive control algorithm. We compare the proposed algorithm in simulation with a model predictive control version of differential dynamic programming.

Subfields: Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

Citations in the corpus

Deep Dynamics Models for Learning Dexterous Manipulation (2019, methodology): “Our method combines components from multiple prior works, including uncertainty estimation [23, 22, 27], deep models and model-predictive control (MPC) [19], and stochastic optimization for planning [28].”
Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning (2018, methodology): “All model-based approaches (MB, MB+DE, GrBAL, and ReBAL) use model bootstrapping, use the same neural network architecture, and use the same planner within experiments: MPPI (Williams et al., 2015) for the simulated experiments and random shooting (RS) (Nagabandi et al., 2017a) for the real-world ex”
Aggressive driving with model predictive path integral control (2016, methodology): “We omit this derivation for brevity and refer the reader to [20].”
Reset-Free Lifelong Learning with Skill-Space Planning (2020, methodology): “At every timestep, MPC selects the policy π that maximizes the predicted H -horizon expected return from the current state s 0 for a speciﬁed reward function r : We use Model Path Predictive Integral (MPPI) (Williams et al., 2015) as our optimizer.”
Autoregressive Dynamics Models for Offline Policy Evaluation and Optimization (2021, methodology): “To take advantage of the dynamics models at test time, we use a variant of Model Predictive Path Integral (MPPI) for planning.”
Learning to Adapt: Meta-Learning for Model-Based Control (2018, methodology): “ng horizon H. We use a planning Hthat is smaller than the adaptation horizon K, since the adapted model is only valid within the current context. We use model predictive path integral control (MPPI) (Williams et al., 2015), but, in principle, our model adaptation approach is agnostic to the model pr”
Planning with Diffusion for Flexible Behavior Synthesis (2022, methodology): “We also investigated a variant using Diffuser as a dynamics model in conventional trajectory optimizers such as MPPI (Williams et al., 2015), but found that this combination performed no better than random, suggesting that the effectiveness of Diffuser stems from coupled modeling and planning, and…”
Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning (2021, methodology): “, 2018) or path integral optimal control (Williams et al., 2015; Lowrey et al., 2019), as well as more sophisticated models and planning or control methods would likely improve overall performance further, but is outside the scope of this work.”
TD-MPC2: Scalable, Robust World Models for Continuous Control (2023, methodology): “Specifically, our approach leverages the MPC framework for local trajectory optimization using Model Predictive Path Integral (MPPI) (Williams et al., 2015) .”
Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion (2024, unknown): “In the spirit of so-called shooting methods like MPPI [63], x k t is then guided by the expected reward over the distribution of all future outcomes instead of one particular outcome.”
Composing Ensembles of Pre-trained Models via Iterative Consensus (2022, methodology): “We use the combination of Model Predictive Control (MPC) (Williams et al., 2015) and the World Model as the generator.”
Temporal Difference Learning for Model Predictive Control (2022, background): “Speciﬁcally, TD-MPC leverages Model Predictive Path Integral (MPPI; Williams et al. (2015)) con-trol for planning (denoted Π θ ), learned models d θ , R θ of the (latent) dynamics and reward signal, respectively, a terminal state-action value function Q θ , and a parameterized policy π θ that helps…”
Text2Motion: from natural language instructions to feasible plans (2023, background): “The average success rates of the converged policies over 100 evaluation episodes are: π Pick with 99%, π Place with 90%, π Pull with 86%, π Push with 97%.”
Model-Based Offline Planning (2020, methodology): “We have considered only physics-bound tasks in this paper as the underlying methods (MPC, MPPI) are known to work well on real systems [5, 2, 21].”