Real-Time Perception Meets Reactive Motion Generation

By Daniel Kappler, F. Meier, Jan Issac, Jim Mainprice, C. Cifuentes, Manuel Wüthrich, Vincent Berenz, Stefan Schaal, Nathan D. Ratliff, Jeannette Bohg. 2017

124 citations recorded by the source corpus.

We address the challenging problem of robotic grasping and manipulation in the presence of uncertainty. This uncertainty is due to noisy sensing, inaccurate models, and hard-to-predict environment dynamics. We quantify the importance of continuous, real-time perception and its tight integration with reactive motion generation methods in dynamic manipulation scenarios. We compare three different systems that are instantiations of the most common architectures in the field: 1) a traditional sense-plan-act approach that is still widely used; 2) a myopic controller that only reacts to local environment dynamics; and 3) a reactive planner that integrates feedback control and motion optimization. All architectures rely on the same components for real-time perception and reactive motion generation to allow a quantitative evaluation. We extensively evaluate the systems on a real robotic platform in four scenarios that exhibit either a challenging workspace geometry or a dynamic environment. We quantify the robustness and accuracy that is due to integrating real-time feedback at different time scales in a reactive motion generation system. We also report on the lessons learned for system building.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Control and Systems Engineering, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Robotics & Control

Citations in the corpus

Habitat 2.0: Training Home Assistants to Rearrange their Habitat (2021, background): “For instance, work in the area of motion planning has made several advancements with new sampling techniques [94, 95] and optimization based methods [86, 87, 89, 96], but largely operated on the assumption of a reliable perception stack.”
Vision-Only Robot Navigation in a Neural Radiance World (2021, unknown): “In [24], the authors combine a new geometric trajectory optimizer [25] with an extension of this SDF obstacle representation that interpolates between grid cells, thereby observing improved optimization [26].”
Learning robust, real-time, reactive robotic grasping (2019, background): “The advantages of visual servoing and other closed-loop methods are that they are able to adapt to dynamic environments and do not necessarily require fully accurate camera calibration or position control (Kappler et al., 2018).”
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects (2023, background): “Computing the rigid 6D transformation from the object to the camera, also known as object pose estimation, is crucial for a variety of applications, such as robotic manipulation [29, 63, 64] and mixed reality [40].”