π0: A Vision-Language-Action Flow Model for General Robot Control

By Kevin Black, Noah Brown, Danny Driess, A. Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter and 14 more. arXiv.org, 2024

2,220 citations recorded by the source corpus.

Robot learning holds tremendous promise to unlock the full potential of flexible, general, and dexterous robot systems, as well as to address some of the deepest questions in artificial intelligence. However, bringing robot learning to the level of generality required for effective real-world systems faces major obstacles in terms of data, generalization, and robustness. In this paper, we discuss how generalist robot policies (i.e., robot foundation models) can address these challenges, and how we can design effective generalist robot policies for complex and highly dexterous tasks. We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge. We then discuss how this model can be trained on a large and diverse dataset from multiple dexterous robot platforms, including single-arm robots, dual-arm robots, and mobile manipulators. We evaluate our model in terms of its ability to perform tasks in zero shot after pre-training, follow language instructions from people and from a high-level VLM policy, and its ability to acquire new skills via fine-tuning. Our results cover a wide variety of tasks, such as laundry folding, table cleaning, and assembling boxes.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/7e7e59d2e247d99954081080ddd5aae93d10b9e0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

GR00T N1: An Open Foundation Model for Generalist Humanoid Robots (2025, positive): “An alternative approach is to finetune pre-trained foundation models on robotics data to build Vision-Language-Action (VLA) models (Black et 2023).”
RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation (2025, neutral): “…and actions in a single transformer for real-time kitchen tasks; RT-2 [2] co-fine-tunes large vision–language models on web and robot data to unlock semantic planning and object reasoning; diffusion-based RDT-1B [26] and the π 0 [1] capture diverse bimanual dynamics from over a million episodes.”
Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success (2025, positive): “Prior works [51, 26, 3] have shown that vanilla LoRA fine-tuning with autoregressive VLAs [22] is impractical for such tasks, as its throughput (3-5 Hz for single-arm robots and even lower for bimanual tasks) falls well below the 25-50 Hz required for real-time deployment.”
SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model (2025, neutral): “Although Spa-tialVLA achieves 21Hz inference speed, it is slower than diffusion decoding [5, 12, 32], which decodes tokens into multiple consecutive actions.”
A Survey on Vision--Language--Action Models for Embodied AI (2024, neutral): “Many have made significant progress [139], [147], but still lacks generalization capability on par with LLMs in NLP. Attaining [231] and LanguageBind [232], align different modalities to the image or language embedding space, respectively.”
SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics (2025, positive): “Unlike prior works relying exclusively on self-attention (SA) (Black et al., 2024) or cross-attention (CA) (Bjorck et al., 2025), we employ an interleaved approach, where each block contains either a CA or a SA layer.”
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning (2025, neutral): “…have been pre-trained on internet scale video and text data, which are then fine-tuned (or adapted) to also predict actions by using behavior cloning from expert demonstrations (Driess et al., 2023; Brohan et al., 2023; Black et al., 2024; Kim et al., 2024; Bjorck et al., 2025; Black et al., 2025).”