Learning descriptors for object recognition and 3D pose estimation

By Paul Wohlhart, Vincent Lepetit. 2015

463 citations recorded by the source corpus.

Detecting poorly textured objects and estimating their 3D pose reliably is still a very challenging problem. We introduce a simple but powerful approach to computing descriptors for object views that efficiently capture both the object identity and 3D pose. By contrast with previous manifold-based approaches, we can rely on the Euclidean distance to evaluate the similarity between descriptors, and therefore use scalable Nearest Neighbor search methods to efficiently handle a large number of objects under a large range of poses. To achieve this, we train a Convolutional Neural Network to compute these descriptors by enforcing simple similarity and dissimilarity constraints between the descriptors. We show that our constraints nicely untangle the images from different objects and different views into clusters that are not only well-separated but also structured as the corresponding sets of poses: The Euclidean distance between descriptors is large when the descriptors are from different objects, and directly related to the distance between the poses when the descriptors are from the same object. These important properties allow us to outperform state-of-the-art object views representations on challenging RGB and RGB-D data.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

OctNet: Learning Deep 3D Representations at High Resolutions (2016, background): “Most existing approaches to 3D pose estimation [3,4,38,43,45] assume that the true 3D shape of the object instance is known.”
Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection (2016, methodology): “However, applications of CNNs to robotic control problems has been less prevalent, compared to applications to passive perception tasks such as object recognition (Krizhevsky et al., 2012; Wohlhart & Lepetit, 2015), localization (Girshick et al., 2014), and segmentation (Chen et al., 2014).”
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks (2016, methodology): “We use a cropped version of the dataset [45], where each image is cropped with one of 11 objects in the center.”
Domain Separation Networks (2016, methodology): “ns for training, 1;280 random GTSRB real–world signs for domain adaptation and validation, and the remaining 37;929 GTSRB real signs as the test set. Synthetic Objects to LineMod. The LineMod dataset [32] consists of CAD models of objects in a cluttered environment and a high variance of 3D poses fo”
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes (2017, methodology): “ufﬁcient textures on the objects in order to compute the local features. To deal with texture-less objects, several methods are proposed to learn feature descriptors using machine learning techniques [32, 10]. A few approaches have been proposed to directly regress to 3D object coordinate location f”
FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation (2021, unknown): “Traditional methods utilize hand-coded [19, 20, 51] templates or features optimized by surrogate objectives [2, 61, 67, 28] from RGBD data and perform correspondence grouping and hypothesis verification.”
Deep Learning in Medical Imaging: General Overview (2017, background): “In recent years, promising results for object matching in computer vision tasks have been reported via ML methods (54–57).”
Learning local feature descriptors with triplets and shallow convolutional neural networks (2016, methodology): “Recent work on deep learning for learning feature embeddings examines the use of triplets of samples instead of solely focusing on pairs [12, 21, 23].”
Synthetic Data for Deep Learning (2019, methodology): “applied this GAN to the Synthetic Cropped LineMod dataset, a synthetic version of a small object classification dataset [638], doing both classification and pose estimation for the objects.”
Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimizing Global Loss Functions (2015, methodology): “, Net(xi,xj)); the classification network in (c) can be used when classes of local image descriptors can be defined [10] and used in a multiclass classification problem; and in (d) the recently proposed triplet network [33, 14, 26, 35] is displayed with different loss functions and input types, wher”
BOP: Benchmark for 6D Object Pose Estimation (2018, background): “[34,24,18,2,36,21,27,25], but it is unclear which methods perform well and in which scenarios.”
Deep Metric Learning with Hierarchical Triplet Loss (2018, background): “…achieved great success in numerous visual understanding tasks, including images or object retrieval [26,30,34], single-shot object classiﬁcation [30,34,32], key-point descriptor learning [12,24], face veriﬁcation [22,20], person re-identiﬁcation [30,23], object tracking [29] and etc. Recently,…”
DeceptionNet: Network-Driven Domain Randomization (2019, background): “The second dataset is the Cropped LineMOD dataset [51] consisting of small centered, cropped 64×64 patches of 11 different objects in cluttered indoor settings displayed in various of poses.”
DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations (2020, methodology): “In the computer vision (CV) literature, deep metric learning (DML) has been used to great effect for learning visual representations [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34].”
Partially Does It: Towards Scene-Level FG-SBIR with Partial Input (2022, background): “Learning unsupervised region-wise correspondence Matching same or similar structure/content from two or more input data is a fundamental task in various downstream applications [48, 82] such as image stitching [10], image fusion [49], co-segmentation [24,77], image retrieval [89], object recognition”
BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth (2017, unknown): “[7, 24] consider a template-based representation computed from RGB-D or RGB data, which allows for large scale detection [11].”
DRIT++: Diverse Image-to-Image Translation via Disentangled Representations (2019, methodology): “rthermore, we demonstrate the potential application of unsupervised domain adaptation. On the tasks of adapting domains from MNIST [24] to MNISTM [12] and Synthetic Cropped LineMod to Cropped LineMod [15,43], we show competitive performance against state-of-the-art domain adaptation methods. We make”
T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-Less Objects (2017, background): “global description relying primarily on intensity edges and depth cues [20, 24, 54, 5, 14, 21, 27].”
DepthSynth: Real-Time Realistic Synthetic Data Generation from CAD Models for 2.5D Recognition (2017, methodology): “As a workaround, some approaches fine-tune their systems on a small set of real scans [52]; but in many cases, access to real data is too”
Smart Mining for Deep Metric Learning (2017, methodology): “The triplet network [24, 6, 14, 25] is trained with an input composed of an anchor point xi (from class yi), another point from the same class x+i = xj (with i 6= j and yi = yj), and a point from a different class xi = xk ((with k 6= i and yi 6= yk).”