The YCB object and Model set: Towards common benchmarks for manipulation research

By Berk Çalli, Arjun Singh, Aaron Walsman, Siddhartha S. Srinivasa, Pieter Abbeel, Aaron M. Dollar. 2015

1,019 citations recorded by the source corpus.

In this paper we present the Yale-CMU-Berkeley (YCB) Object and Model set, intended to be used for benchmarking in robotic grasping and manipulation research. The objects in the set are designed to cover various aspects of the manipulation problem; it includes objects of daily life with different shapes, sizes, textures, weight and rigidity, as well as some widely used manipulation tests. The associated database provides high-resolution RGBD scans, physical properties and geometric models of the objects for easy incorporation into manipulation and planning software platforms. A comprehensive literature survey on existing benchmarks and object datasets is also presented and their scope and limitations are discussed. The set will be freely distributed to research groups worldwide at a series of tutorials at robotics conferences, and will be otherwise available at a reasonable purchase cost.

Subfields: Control and Systems Engineering, Industrial and Manufacturing Engineering, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Industrial and Manufacturing Engineering

In Evaluation & Benchmarks

Citations in the corpus

Domain randomization for transferring deep neural networks from simulation to the real world (2017, methodology): “To test the performance of our object detectors on real-world objects with non-uniform textures, we trained an object detector to localize a can of Spam from the YCB Dataset [3].”
Habitat 2.0: Training Home Assistants to Rearrange their Habitat (2021, methodology): “[Yes] Cited [63] for use of the YCB objects.”
Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning (2019, background): “There is a long history of robotics benchmarks [28], datasets [29, 30, 31, 32, 33, 34, 35], competitions [36] and standardized object sets [37, 38] that have played an important role in robotics research.”
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes (2017, methodology): “The objects we used are a subset of 21 of the YCB objects [1], selected due to high-quality 3D models and good visibility in depth.”
Yale-CMU-Berkeley dataset for robotic manipulation research (2017, background): “Nevertheless, for detailed information about the YCB benchmarking effort in general, we refer the reader to Calli et al. (2015b), which focuses on the criteria for choosing the objects and their utilization for benchmarking in robotics.”
Reconstructing Hand-Object Interactions in the Wild (2020, methodology): “Two primary sources of 3D object models we use are the YCB dataset [4] and the Free3D online platform.”
Rubik's Cube Handling Using a High-Speed Multi-Fingered Hand and a High-Speed Vision System (2018, background): “1: High-speed robotic hand and Rubik’s cube the Task Items of The YCB Object and Model Set, which is considered to be a suitable benchmark for continuous operation [5].”
PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation (2019, neutral): “YCB-Video Dataset contains 21 YCB [4] objects of varying shape and texture.”
Correcting Robot Plans with Natural Language Feedback (2022, background): “Each environment contained two objects from a set of four YCB objects [11]: a Cheeze-it box, a bleach bottle, a can of spam and a bottle of mustard.”
Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects (2018, methodology): “Since our goal is robotic manipulation of the objects whose 6-DoF pose has been estimated, we leverage the popular YCB objects [18].”
Evaluating Real-World Robot Manipulation Policies in Simulation (2024, neutral): “Simulated evaluations have great potential as a tool for practitioners: they can reduce the need for costly real-world evaluations and provide a more comprehensive picture of a policy’s performance by enabling sweeps across a wide range of controlled environment conditions at negligible added cost.”
Alternative Paths Planner (APP) for Provably Fixed-time Manipulation Planning in Semi-structured Environments (2020, methodology): “The goal is for the robot to pick up the red bowl, while avoiding two obstacles: large blue pitchers from the YCB object set [17].”
GanHand: Predicting Human Grasp Affordances in Multi-Object Scenes (2020, methodology): “Our dataset, called YCB-Affordance, features grasps for all objects from the YCB Object set [11] for which a CAD model was available.”
Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation (2019, methodology): “Up to 5 objects are randomly drawn from a subset of the YCB Dataset [17] and placed on the table.”
DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion (2019, neutral): “[5] of varying”
Masked Visual Pre-training for Motor Control (2022, methodology): “We import various objects from the YCB dataset (Calli et al., 2015)—box, can, mug, and banana—for the pick task and re-train the model for each individual object.”
Shape completion enabled robotic grasping (2016, methodology): “The Holdout Models were sampled from holdout YCB and Grasp Dataset objects.”
TD-MPC2: Scalable, Robust World Models for Continuous Control (2023, background): “Pick YCB is the hardest task and considers manipulation of all 74 objects from the YCB (Calli et al., 2015) dataset.”