Three-Dimensional Object Recognition from Single Two-Dimensional Images

By David G. Lowe. 1987

Abstract A computer vision system has been implemented that can recognize three-dimensional objects from unknown viewpoints in single gray-scale images. Unlike most other approaches, the recognition is accomplished without any attempt to reconstruct depth information bottom-up from the visual input. Instead, three other mechanisms are used that can bridge the gap between the two-dimensional image and knowledge of three-dimensional objects. First, a process of perceptual organization is used to form groupings and structures in the image that are likely to be invariant over a wide range of viewpoints. Second, a probabilistic ranking method is used to reduce the size of the search space during model-based matching. Finally, a process of spatial correspondence brings the projections of three-dimensional models into direct correspondence with the image by solving for unknown viewpoint and model parameters. A high level of robustness in the presence of occlusion and missing data can be achieved through full application of a viewpoint consistency constraint. It is argued that similar mechanisms and constraints form the basis for recognition in human vision.

Subfields: Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

Domain randomization for transferring deep neural networks from simulation to the real world (2017, methodology): “Researchers in computer vision have used 3D models as a tool to improve performance on real images since the earliest days of the ﬁeld (e.g., [30], [24]).”
MarrNet: 3D Shape Reconstruction via 2.5D Sketches (2017, background): “Researchers have explored this idea for decades [Lowe, 1987].”
Lifting Object Detection Datasets into 3D (2015, background): “In certain cases a precise 3D model of the target object is known [22], [23] and the problem can be seen as a special case of class-based reconstruction that focuses on reconstructing a speciﬁc instance of the class.”
Single Image 3D Interpreter Network (2016, methodology): “culated objects [47,57,1], like a human body or the base of a swivel chair. Typically, researchers recovered a 3D skeleton from a single image by minimizing its projection error on the 2D image plane [27,22,12,55,33,62]. Recent work in this line [1,61] demonstrated state-of-the-art performance. In c”
Learning 3D Object Categories by Looking Around Them (2017, methodology): “A traditional approach to 3D reconstruction is to use handcrafted 3D models [29, 21], and more recently 3D CAD models [5, 43].”
The MOPED framework: Object recognition and pose estimation for manipulation (2011, background): “Solutions such as grouping (Lowe 1987), interpretation trees (Grimson 1991) or image space clustering (Collet et al. 2009) are often used, but false positives often arise from algorithms not being able to handle unexpected scene complexity.”
Visual servo control. I. Basic approaches (2006, background): “While this problem is beyond the scope of the present tutorial, many solutions have been presented in the literature (see, e.g., [16] and [17]).”
Enriching object detection with 2D-3D registration and continuous viewpoint estimation (2015, background): “Interestingly, the problem of matching 3D models to 2D images has been explored since the early days of computer vision [15], but had largely been neglected in recent years in favor of 2D detectors based on robust local features and statistical learning techniques.”
Real-time line detection through an improved Hough transform voting scheme (2008, methodology): “In our current implementation of the described algorithm, we have chosen to group approximately collinear feature pixels using the linking procedure from [26] and the subdivision scheme from [27] because they are fast and handle the general case.”
Object recognition from local scale-invariant features (1999, background): “This process is presumably playing the same role in verification as the parameter solving and outlier detection used in this paper, since the accuracy of interpretations can often depend on enforcing a single viewpoint constraint [11].”
View-Invariant Recognition Using Corresponding Object Fragments (2004, methodology): “[6]). Due to this requirement, recognition using 3D data is frequently considered separately from image-based methods. For examples of 3D approaches, see [7, 8 ].”
Local feature view clustering for 3D object recognition (2001, background): “It is not sufficient to use a metric such as just the number or type of feature matches [3, 5] to determine the presence of a model under these circumstances.”
Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense (2020, background): “Instead, they use priors—groupings and structural cues [211,212] that are likely to be invariant over wide ranges of viewpoints [213]—resulting in feature-based approaches [87,214].”