Analysis and Observations From the First Amazon Picking Challenge

By Nikolaus Correll, Kostas E. Bekris, Dmitry Berenson, Oliver Brock, Albert J. Causo, Kris K. Hauser, K. Okada, Alberto Rodriguez, Joseph M. Romano, Peter R. Wurman. 2016

458 citations recorded by the source corpus.

This paper presents an overview of the inaugural Amazon Picking Challenge along with a summary of a survey conducted among the 26 participating teams. The challenge goal was to design an autonomous robot to pick items from a warehouse shelf. This task is currently performed by human workers, and there is hope that robots can someday help increase efficiency and throughput while lowering cost. We report on a 28-question survey posed to the teams to learn about each team’s background, mechanism design, perception apparatus, planning, and control approach. We identify trends in this data, correlate it with each team’s success in the competition, and discuss observations and lessons learned based on survey results and the authors’ personal experiences during the challenge.Note to Practitioners—Perception, motion planning, grasping, and robotic system engineering have reached a level of maturity that makes it possible to explore automating simple warehouse tasks in semistructured environments that involve high-mix, low-volume picking applications. This survey summarizes lessons learned from the first Amazon Picking Challenge, highlighting mechanism design, perception, and motion planning algorithms, as well as software engineering practices that were most successful in solving a simplified order fulfillment task. While the choice of mechanism mostly affects execution speed, the competition demonstrated the systems challenges of robotics and illustrated the importance of combining reactive control with deliberative planning.

Subfields: Control and Systems Engineering, Industrial and Manufacturing Engineering, Mechanical Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Industrial and Manufacturing Engineering

In Mechanical Engineering

Citations in the corpus

Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning (2019, background): “There is a long history of robotics benchmarks [28], datasets [29, 30, 31, 32, 33, 34, 35], competitions [36] and standardized object sets [37, 38] that have played an important role in robotics research.”
Learning Visual Affordances for Robotic Manipulation (2019, background): “Existing frameworks such as LINEMOD [41] or MOPED [17] work well under certain assumptions such as objects sitting on a table top with good illumination, but underperform when confronted with the limited visibility, shadows, and clutter imposed by the APC scenario [18].”
Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching (2017, background): “Unlike earlier versions of the competition [36], half of the objects were novel in the 2017 edition of the competition.”
Evaluating Real-World Robot Manipulation Policies in Simulation (2024, unknown): “Yet, we emphasize that our goal is not to completely replace real-world evaluations: a simulator will always be an imperfect proxy for the real world.”
CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation (2024, unknown): “While standardized real-world setups can be beneficial, they often require substantial time and resources [10, 16, 31, 76].”
GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger (2018, background): “In cluttered environments like a pantry or a warehouse storage cell, as in the Amazon Robotics Challenge [7], a robot faces the challenge of singulating target objects from a tightly-packed collection of items.”
Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge (2016, background): “Existing frameworks such as LINEMOD [12] or MOPED [13] work well under certain assumptions such as objects sitting on a table top with good illumination, but underperform when confronted with the limited visibility, shadows, and clutter imposed by the APC scenario [14].”
PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation (2018, background): “For instance, fast and robust pose estimation is crucial in Amazon Picking Challenge [6], where a robot needs to pick objects from a warehouse shelf.”
Real-Time Perception Meets Reactive Motion Generation (2017, background): “The teams who participated in the recent robotics challenges (DARPARobotics Challenge (DRC) [3], [4] or Amazon Picking Challenge (APC) [5], [6]) testified to the importance of fast perceptual feedback integrated into planning and control loops.”
A self-supervised learning system for object detection using physics simulation and multi-view pose estimation (2017, background): “Nevertheless, in application domains, such as those studied by the Amazon Picking Challenge [2], which involve varying light conditions and cluttered scenes, it has been shown [5] that CNN-based segmentation [10], [15] followed by point cloud registration with 3D models [23], [24], [25] is an effect”
Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline (2021, background): “Processing 3D point cloud data accurately is crucial in many applications including autonomous driving (Navarro-Serment et al., 2010; Kidono et al., 2011), robotics (Rusu et al., 2009; Correll et al., 2016; Mousavian et al., 2019) and scene understanding (Aldoma et al., 2012).”
Dex-Net 3.0: Computing Robust Robot Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning (2017, methodology): “End-effectors based on suction are widely used in industrial applications such as warehouse order fulfillment, handling limp materials such as fabric [19] or produce [27], and robot applications such as the Amazon Picking Challenge [4], underwater manipulation [34], or wall climbing [2].”
Visual Room Rearrangement (2021, background): “…complete observability of the state from perfect visual perception [11, 27], a planar surface as the environment [30, 42], a static robot [15, 32], same environment for evaluation of generalization [39, 26], or a limited set of object categories or limited variation within the categories [10, 19].”