OpenVLA: An Open-Source Vision-Language-Action Model

By Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, E. Foster, Grace Lam, Pannag R. Sanketi and 8 more. Conference on Robot Learning, 2024

2,794 citations recorded by the source corpus.

Large policies pretrained on a combination of Internet-scale vision-language data and diverse robot demonstrations have the potential to change how we teach robots new skills: rather than training new behaviors from scratch, we can fine-tune such vision-language-action (VLA) models to obtain robust, generalizable policies for visuomotor control. Yet, widespread adoption of VLAs for robotics has been challenging as 1) existing VLAs are largely closed and inaccessible to the public, and 2) prior work fails to explore methods for efficiently fine-tuning VLAs for new tasks, a key component for adoption. Addressing these challenges, we introduce OpenVLA, a 7B-parameter open-source VLA trained on a diverse collection of 970k real-world robot demonstrations. OpenVLA builds on a Llama 2 language model combined with a visual encoder that fuses pretrained features from DINOv2 and SigLIP. As a product of the added data diversity and new model components, OpenVLA demonstrates strong results for generalist manipulation, outperforming closed models such as RT-2-X (55B) by 16.5% in absolute task success rate across 29 tasks and multiple robot embodiments, with 7x fewer parameters. We further show that we can effectively fine-tune OpenVLA for new settings, with especially strong generalization results in multi-task environments involving multiple objects and strong language grounding abilities, and outperform expressive from-scratch imitation learning methods such as Diffusion Policy by 20.4%. We also explore compute efficiency; as a separate contribution, we show that OpenVLA can be fine-tuned on consumer GPUs via modern low-rank adaptation methods and served efficiently via quantization without a hit to downstream success rate. Finally, we release model checkpoints, fine-tuning notebooks, and our PyTorch codebase with built-in support for training VLAs at scale on Open X-Embodiment datasets.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/8f9ceb5ffad8e7a066dfc9d9aaa5153b714740ee

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. Chelsea Finn
3. Pieter Abbeel
4. Trevor Darrell
5. Mohit Bansal
6. Dacheng Tao
7. Quoc V. Le
8. Percy Liang

Citations in the corpus

• Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces (2024, positive): “…82] and the feature extraction abilities of modern vision encoders [30, 65, 69], MLLMs, especially visual MLLMs, exhibit unprecedented visual understanding capabilities [34, 40, 78, 86, 91, 102, 103], a promising direction toward developing world models [50] and embodied agents [17, 21, 37, 59].”
• TinyVLA: Toward Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation (2024, neutral): “Baseline: We evaluated our method against Diffusion Policy (DP) [3], Multimodal Diffusion [38] and OpenVLA [10].”
• RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation (2025, positive): “Vision–language–action (VLA) frameworks like OpenVLA [19] and CogACT [21], together with adaptations like Octo [34], LAPA [41], and OpenVLA-OFT [18] demonstrate efficient fine-tuning to novel robots and sensor modalities.”
• AgiBot World Colosseo: A Large-Scale Manipulation Platform for Scalable and Intelligent Embodied Systems (2025, positive): “Since the latent action space is orders of magnitude smaller than the discretized low-level actions used in OpenVLA [4], this approach also facilitates the efficient adaptation of general-purpose VLMs into robot policies.”
• CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models (2025, positive): “Following [26], we evaluate on four tasks designed in [26] to evaluate visual robustness (varying distractors), motion generalization (novel object positions), semantic generalization (unseen language instruction), and language grounding (instruction following).”
• CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation (2024, positive): “[7], RT-1-X [48], RT-2-X [48], Octo [62], and OpenVLA [30] across simulated benchmarks (first thee charts) and real-world evaluations (last three charts), using various robots: Google robot, WidowX robot, Realman robot, and Franka robot.”
• π0: A Vision-Language-Action Flow Model for General Robot Control (2024, neutral): “We compare to OpenVLA [24], a 7B parameter VLA model that was originally trained on the OXE dataset [10].”
• A Survey on Vision--Language--Action Models for Embodied AI (2024, positive): “DINOv2 learns both pixel-and image-level features by combining masked image modeling with a momentum encoder and multi-crop augmentation, with benefits on downstream robotic tasks evidenced by OpenVLA [50].”
• RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation (2024, positive): “Inspired by recent attempts in unimanual manipulation (Brohan et al., 2023; Kim et al., 2024), we mitigate this through cross-robot pretraining: leveraging multi-robot datasets for pre-training followed by target-robot fine-tuning, amplifying data volume by three orders of magnitude to extract…”
• UniVLA: Learning to Act Anywhere with Task-centric Latent Actions (2025, positive): “In contrast, some prior works [10, 39, 46] leverage pretrained VLMs to generate robotic actions by tapping into world knowledge from large-scale vision-language datasets.”
• Robotic Control via Embodied Chain-of-Thought Reasoning (2024, positive): “We evaluate on a 6-DoF WidowX robot arm, a commonly-used setup for evaluating learned robot policies [16, 7].”
• SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model (2025, positive): “Specifically, we use ZoeDepth [4] to estimate depth map D and obtain pixel’s 3D position p = { x, y, z } in the egocentric 3D coordinate system via back-projection π − 1 with camera intrinsic parameters.”
• SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics (2025, positive): “The dataset used for training contains approximately 23k trajectories, and is significantly smaller than those used in typical VLA training regimes–OpenVLA, for instance, utilizes around 1 million trajectories.”
• V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning (2025, neutral): “…have been pre-trained on internet scale video and text data, which are then fine-tuned (or adapted) to also predict actions by using behavior cloning from expert demonstrations (Driess et al., 2023; Brohan et al., 2023; Black et al., 2024; Kim et al., 2024; Bjorck et al., 2025; Black et al., 2025).”
• π0.5: a Vision-Language-Action Model with Open-World Generalization (2025, positive): “VLA policies are typically initialized from a pre-trained vision-language model, which has been exposed to large amounts of internet vision and language data [23, 92, 42].”
• Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success (2025, positive): “While Kim et al. [22] study various parameter update strategies and from their findings show that LoRA fine-tuning enables effective adaptation to single-arm robots operating at low control frequencies ( < 10 Hz), their analysis does not extend to bimanual robots with high control frequencies…”