Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets

By Frederik D. Ebert, Yanlai Yang, Karl Schmeckpeper, Bernadette Bucher, Georgios Georgakis, Kostas Daniilidis, Chelsea Finn, Sergey Levine. Robotics: Science and Systems Conference, 2021

365 citations recorded by the source corpus.

Robot learning holds the promise of learning policies that generalize broadly. However, such generalization requires sufficiently diverse datasets of the task of interest, which can be prohibitively expensive to collect. In other fields, such as computer vision, it is common to utilize shared, reusable datasets, such as ImageNet, to overcome this challenge, but this has proven difficult in robotics. In this paper, we ask: what would it take to enable practical data reuse in robotics for end-to-end skill learning? We hypothesize that the key is to use datasets with multiple tasks and multiple domains, such that a new user that wants to train their robot to perform a new task in a new domain can include this dataset in their training process and benefit from cross-task and cross-domain generalization. To evaluate this hypothesis, we collect a large multi-domain and multi-task dataset, with 7,200 demonstrations constituting 71 tasks across 10 environments, and empirically study how this data can improve the learning of new tasks in new environments. We find that jointly training with the proposed dataset and 50 demonstrations of a never-before-seen task in a new domain on average leads to a 2x improvement in success rate compared to using target domain data alone. We also find that data for only a few tasks in a new domain can bridge the domain gap and make it possible for a robot to perform a variety of prior tasks that were only seen in other domains. These results suggest that reusing diverse multi-task and multi-domain datasets, including our open-source dataset, may pave the way for broader robot generalization, eliminating the need to re-collect data for each new robot learning project.

View Paper: https://www.semanticscholar.org/paper/6bae20930eaa0d9d489317f3b3b1aaaf18205ef8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Learning Interactive Real-World Simulators (2023, background): “An increasing amount of video data of real-robot executions paired with task descriptions such as the Bridge Data [18] and data that enabled RT-1 and RT-2 [19] are becoming increasingly available.”
AgiBot World Colosseo: A Large-Scale Manipulation Platform for Scalable and Intelligent Embodied Systems (2025, unknown): “While significant progress has been made in general-purpose foundational models for natural language processing [1] and computer vision [2], robotics lags behind due to the difficulty of (high-quality) data collection.”
Learning Universal Policies via Text-Guided Video Generation (2023, methodology): “The robotic dataset is adopted from the Bridge dataset (Ebert et al., 2021) with 7.2k video-text pairs, where we use the task IDs as texts.”
TinyVLA: Toward Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation (2024, unknown): “T RAINING multitasking robot imitators to operate in complex and uncertain environments faces considerable challenges due to limited data and the difﬁculty of learning physical motion [1], [2].”
SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics (2025, unknown): “In particular, robotic policies (Zhao et al., 2023; Chi et al., 2024; Lee et al., 2024; Hansen et al., 2022) still face challenges in generalizing across object types, positions, environments, and tasks (Xie et al., 2024; Ebert et al., 2021).”
π0: A Vision-Language-Action Flow Model for General Robot Control (2024, unknown): “More recently, a number of high-quality datasets have been collected for robot control that enable broad generalization [23, 10, 52, 33, 34, 43, 13, 6], but typically for simpler tasks that consist of object relocation and rudimentary furniture manipulation (e.g., drawer opening) [31, 15].”
Language to Rewards for Robotic Skill Synthesis (2023, methodology): “To extend the end-to-end approaches to manipulation, a common approach is to utilize latent embeddings of language commands as multitask input context, and train with behavioral cloning [14, 32, 16], offline reinforcement learning [33], goal-conditioned reinforcement learning [34], or in a shared au”
WorldSimBench: Towards Video Generation Models as World Simulators (2024, background): “…advancement of diffusion transformer (Peebles & Xie, 2023) and the extensive utilization of large-scale internet video datasets (Bain et al., 2021; Ebert et al., 2021; Goyal et al., 2017; Grauman et al., 2022), certain Predictive Actionable Video Model (Yang et al., 2023) models, also known as…”
Octo: An Open-Source Generalist Robot Policy (2024, methodology): “While a number of prior works have proposed other pretraining schemes for imitation finetuning [25, 24, 26], to our knowledge no prior method provides a pretrained policy that has been demonstrated to finetune successfully to new observation and action spaces.”
DayDreamer: World Models for Physical Robot Learning (2022, background): “One class of methods attempts to scale up experience collection through a ﬂeet of robots (Kalashnikov et al., 2018; 2021; Ebert et al., 2021; Dasari et al., 2019; Levine et al., 2018).”
R3M: A Universal Visual Representation for Robot Manipulation (2022, methodology): “Many of these works focus on collecting and learning from robot data itself [55, 56, 7, 8, 9, 10, 57].”
Robotic Control via Embodied Chain-of-Thought Reasoning (2024, background): “Towards this goal, recent works have explored training “generalist robot policies” [10, 11, 12, 13, 14, 15, 16, 17] on diverse robot datasets [18, 19, 10, 20, 21, 22, 23, 24, 13, 14, 25, 15, 26, 6].”
MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023, background): “One popular data source is human demonstrators that teleoperate robot arms [2–6, 23– 27 ], but collecting 3 Problem Setup Imitation Learning.”