TD-MPC2: Scalable, Robust World Models for Continuous Control

By Nicklas Hansen, Hao Su, Xiaolong Wang. International Conference on Learning Representations, 2023

506 citations recorded by the source corpus.

TD-MPC is a model-based reinforcement learning (RL) algorithm that performs local trajectory optimization in the latent space of a learned implicit (decoder-free) world model. In this work, we present TD-MPC2: a series of improvements upon the TD-MPC algorithm. We demonstrate that TD-MPC2 improves significantly over baselines across 104 online RL tasks spanning 4 diverse task domains, achieving consistently strong results with a single set of hyperparameters. We further show that agent capabilities increase with model and data size, and successfully train a single 317M parameter agent to perform 80 tasks across multiple task domains, embodiments, and action spaces. We conclude with an account of lessons, opportunities, and risks associated with large TD-MPC2 agents. Explore videos, models, data, code, and more at https://tdmpc2.com

Subfields: Control and Systems Engineering, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/8eeccf1033fd759c868b29ccb6aa70f5ffd1887c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

1. Roberto Mart'in-Mart'in
2. Peter Stone
3. Rohan Chandra
4. Ben Abbatematteo
5. Chen Tang
6. Jiaheng Hu
7. Faraz Torabi
8. W. B. Knox

In Management Science and Operations Research

1. Roberto Mart'in-Mart'in
2. Peter Stone
3. Stefano V. Albrecht

Citations in the corpus

• V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning (2025, unknown): “In particular, we share reported results of VideoMAEv2 (Wang et al., 2023), InternVideo-1B and 6B (Wang et al., 2024b), and VideoPrism (Zhang et al., 2024c) on the classification tasks we consider, when available.”
• Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, methodology): “Combining model-free and model-based approaches is another promising direction to derive sample-efficient RL algorithms (194).”
• Mastering diverse control tasks through world models (2025, unknown): “For continuous control, we run the official implementation of TD-MPC2 43 from proprioceptive inputs and from images.”