The MOPED framework: Object recognition and pose estimation for manipulation

By Alvaro Collet, Manuel Martinez, Siddhartha S. Srinivasa. 2011

430 citations recorded by the source corpus.

We present MOPED, a framework for Multiple Object Pose Estimation and Detection that seamlessly integrates single-image and multi-image object recognition and pose estimation in one optimized, robust, and scalable framework. We address two main challenges in computer vision for robotics: robust performance in complex scenes, and low latency for real-time operation. We achieve robust performance with Iterative Clustering Estimation (ICE), a novel algorithm that iteratively combines feature clustering with robust pose estimation. Feature clustering quickly partitions the scene and produces object hypotheses. The hypotheses are used to further refine the feature clusters, and the two steps iterate until convergence. ICE is easy to parallelize, and easily integrates single- and multi-camera object recognition and pose estimation. We also introduce a novel object hypothesis scoring function based on M-estimator theory, and a novel pose clustering algorithm that robustly handles recognition outliers. We achieve scalability and low latency with an improved feature matching algorithm for large databases, a GPU/CPU hybrid architecture that exploits parallelism at all levels, and an optimized resource scheduler. We provide extensive experimental results demonstrating state-of-the-art performance in terms of recognition, scalability, and latency in real-world robotic applications.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Domain randomization for transferring deep neural networks from simulation to the real world (2017, background): “, a full 3D mesh model [47] or a 3D metric feature representation [6]).”
Learning human activities and object affordances from RGB-D videos (2012, background): “…et al., 2011b; Rosman and Ramamoorthy, 2011; Anand et al., 2012), developing frameworks for object recognition and pose estimation for manipulation (Collet et al., 2011), object tracking for 3D object modeling (Krainin et al., 2011), etc. Robots are now becoming more integrated in human…”
Deep learning for detecting robotic grasps (2013, background): “Each layer (cid:96) will have a set of weights W [ (cid:96) ] mapping from its inputs to its hidden units, so the parameters of our model are Θ = { W [1] , W [2] , W [3] } .”
Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection (2017, methodology): “Early approaches, such as [6], heavily depend on extracting local features such as SIFT [30], SURF [3], MSER [32] and matching them to retrieve instances [29, 48].”
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes (2017, background): “Traditionally, the problem of 6D object pose estimation is tackled by matching feature points between 3D models and images [3], [4], [5].”
FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation (2021, neutral): “6D Pose Estimation is an important component in lots of real-world applications, such as augmented reality [40], autonomous driving [13, 8, 69] and robotic grasping [9, 62, 17].”
PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation (2019, background): “It is an important component in many real-world applications, such as robotic grasping and manipulation [6, 48, 55], autonomous driving [11, 5, 53], augmented reality [31] and so on.”
GDRNPP: A Geometry-Guided and Fully Learning-Based Object Pose Estimator (2021, neutral): “It has wide applicability to many real-world tasks such as robotic manipulation [1], [2], [3], augmented reality [4], [5] and autonomous driving [6], [7].”
PR2 looking at things — Ensemble learning for unstructured information processing with Markov logic networks (2014, background): “Successful examples of such robot object perception systems are described in [9] using point features of 3D opaque objects, or MOPED, which uses visual keypoint descriptors for the learned textured objects [10] and specialized perception systems for translucent objects [11].”
DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion (2019, methodology): “Classical methods rely on detecting and matching keypoints with known object models [1, 7, 9, 27, 44].”
6-DoF object pose from semantic keypoints (2017, background): “To relax the assumption of known 2D landmarks, a number of approaches [5], [6], [7] have considered the detection of discriminative image keypoints, such as SIFT [8], with highly textured objects.”
Glimpse: Continuous, Real-Time Object Recognition on Mobile Devices (2015, methodology): “Feature extraction: This stage processes the content inside the bounding box computed by the detector to extract features that represent the object, using methods like the scale-invariant feature transform (SIFT) [34, 12] and speeded-up robust features (SURF) [7, 12].”
iNeRF: Inverting Neural Radiance Fields for Pose Estimation (2020, methodology): “Classical methods for object pose estimation address the task by detecting and matching keypoints with known 3D models [1], [4], [5], [29].”
Contextually Guided Semantic Labeling and Search for 3D Point Clouds (2011, background): “In the past, a signiﬁcant amount of work has been done in semantic labeling of 2D images (Murphy et al., 2003; Heitz and Koller, 2008; Felzenszwalb et al., 2008; Collet et al., 2011; Li et al., 2012).”
BigBIRD: A large-scale 3D database of object instances (2014, background): “Furthermore, although most recent instance recognition systems work with RGB-D data, there exist high-quality instance recognition systems that use only RGB images, such as MOPED, presented by Collet et al. [19].”
3D-FUTURE: 3D Furniture Shape with TextURE (2020, methodology): “Typical methods 6DoF pose estimation ﬁrst build point-wise correspondences between 3D models and 2D images, followed by the Perspective-n-Point (PnP) algorithm to compute pose parameters (Collet et al., 2011; Fig.”
“Hands On” Visual Recognition for Visually Impaired Users (2017, background): “…2013; Jafri et al. 2013; Nanayakkara et al. 2013; Yi et al. 2014; Shilkrot et al. 2014; Fusco et al. 2015); object recognition (Sudol et al. 2010; Schauerte et al. 2012; Collet et al. 2011; Chincha and Tian 2011; Winlock et al. 2010); and multiple object detection (Mekhalfi et al. 2015a, 2015b).”
Form2Fit: Learning Shape Priors for Generalizable Assembly from Disassembly (2019, methodology): “Classic methods for assembly are often characterized by a perception module that first estimates the 6D object poses of observed parts [1], [2], [3], [4], [5], followed by a planner that then optimizes for picking and placing actions based on the inferred object poses and task-informed goal states.”
PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation (2017, positive): “These include keypoint matching between 2D images and their corresponding 3D CAD models [1, 5, 37], or aligning 3D-reconstructed models with ground-truth models to recover the object poses [28, 9].”