Octo: An Open-Source Generalist Robot Policy

By O. Team, Dibya Ghosh, Homer Rich Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu and 8 more. Robotics: Science and Systems Conference, 2024

1,572 citations recorded by the source corpus.

Large policies pretrained on diverse robot datasets have the potential to transform robotic learning: instead of training new policies from scratch, such generalist robot policies may be finetuned with only a little in-domain data, yet generalize broadly. However, to be widely applicable across a range of robotic learning scenarios, environments, and tasks, such policies need to handle diverse sensors and action spaces, accommodate a variety of commonly used robotic platforms, and finetune readily and efficiently to new domains. In this work, we aim to lay the groundwork for developing open-source, widely applicable, generalist policies for robotic manipulation. As a first step, we introduce Octo, a large transformer-based policy trained on 800k trajectories from the Open X-Embodiment dataset, the largest robot manipulation dataset to date. It can be instructed via language commands or goal images and can be effectively finetuned to robot setups with new sensory inputs and action spaces within a few hours on standard consumer GPUs. In experiments across 9 robotic platforms, we demonstrate that Octo serves as a versatile policy initialization that can be effectively finetuned to new observation and action spaces. We also perform detailed ablations of design decisions for the Octo model, from architecture to training data, to guide future research on building generalist robot models.

Subfields: Computational Theory and Mathematics, Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/1d2753d74025e7a71594506623be81f18b073adb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Computer Networks and Communications

Citations in the corpus

DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset (2024, neutral): “The success of transformer models [58] in natural language processing and computer vision motivated a number of recent works that collected large-scale demonstration datasets and trained transformer-based policies on them [2, 16, 25, 38, 39, 42, 49, 51, 65, 67].”
Autoregressive Image Generation without Vector Quantization (2024, background): “In those scenarios, the distribution of taking an action is formulated as a denoising process on the robot observations, which can be pixels or latents [8, 34].”
TinyVLA: Toward Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation (2024, neutral): “Octo [28] uses cross-embodiment data for pertaining.”
A Survey on Vision--Language--Action Models for Embodied AI (2024, neutral): “While discrete action has demonstrated superior performance in RT-1 [114], Octo [134] argues that it leads to early grasping issues.”
AgiBot World Colosseo: A Large-Scale Manipulation Platform for Scalable and Intelligent Embodied Systems (2025, neutral): “The widely accepted reason is the lack of high-quality data—unlike images and text, which are abundant and standardized, robotic datasets suffer from fragmented clips due to heterogeneous hardware and unstandardized collection procedure, leading to low-quality and inconsistent outcome.”
OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning (2024, background): “Recent efforts [40, 41, 42, 43, 44, 45, 46, 47] have focused on collecting robotic data using various embodiments for different tasks.”
RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation (2025, neutral): “Vision–language–action (VLA) frameworks like OpenVLA [19] and CogACT [21], together with adaptations like Octo [34], LAPA [41], and OpenVLA-OFT [18] demonstrate efficient fine-tuning to novel robots and sensor modalities.”
SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model (2025, neutral): “XIV present comprehensive evaluation results comparing the performance of four state-of-the-art approaches: Diffusion Policy [12], Octo [48], Open-VLA [30], and our proposed SpatialVLA on the Franka Panda Emika robot.”
π0.5: a Vision-Language-Action Model with Open-World Generalization (2025, neutral): “…from narrow, single-task datasets to diverse datasets that span many scenes and tasks [17, 25, 80, 63, 41, 6, 30, 67, 1] allows the resulting policies to not only solve a wider range of tasks out of the box, but also improves their ability to generalize to new scenes and tasks [9, 63, 62, 22].”
CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models (2025, positive): “Some approaches incorporate VLMs [3, 44, 58] into end-to-end trainable policies by using them as pretrained backbone for better visuo-language representation [10, 26, 38, 43, 53].”
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning (2025, positive): “Models are ViT-L/16 trained for 90K iterations using a cosine learning schedule following Bardes et al. (2024).”
Robotic Control via Embodied Chain-of-Thought Reasoning (2024, neutral): “We also compare against Octo [16], which is also trained on that dataset, but was not fine-tuned from a VLM (i.e., it is not a VLA).”
Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success (2025, neutral): “However, for broader comparison, we also include LIBERO results from prior state-of-the-art imitation learning methods, such as Diffusion Policy [5], Octo [47], DiT Policy [13], Seer [48], MDT [38], and π 0 [3].”
Evaluating Real-World Robot Manipulation Policies in Simulation (2024, neutral): “Instead, our aim is to give practitioners a readily available signal for policy improvement to guide their research.”
Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation (2024, background): “Recently, co-training on diverse real-world datasets collected from different but similar types of robots have shown promising results on single-arm manipulation [11, 20, 31, 61, 98], and on navigation [79].”
CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation (2024, neutral): “For Octo [62] and OpenVLA [30], we tested multiple finetuned checkpoints at different steps and select the ones that worked best in the real-world tasks.”
π0: A Vision-Language-Action Flow Model for General Robot Control (2024, neutral): “We also compare to Octo [50], a smaller 93M parameter model.”
OpenVLA: An Open-Source Vision-Language-Action Model (2024, methodology): “Generalist Robot Policies A recent trend in robotics works towards training multi-task “generalist” robot policies [2, 6, 45–49] on large diverse robot datasets [1, 2, 6, 11, 45, 49–56], spanning many different robot embodiments [1, 5, 53, 57–66].”