Example Based 3D Reconstruction from Single 2D Images

By Tal Hassner, Ronen Basri. 2006

140 citations recorded by the source corpus.

We present a novel solution to the problem of depth reconstruction from a single image. Single view 3D reconstruction is an ill-posed problem. We address this problem by using an example-based synthesis approach. Our method uses a database of objects from a single class (e.g. hands, human figures) containing example patches of feasible mappings from the appearance to the depth of each object. Given an image of a novel object, we combine the known depths of patches from similar objects to produce a plausible depth estimate. This is achieved by optimizing a global target function representing the likelihood of the candidate depth. We demonstrate how the variability of 3D shapes and their poses can be handled by updating the example database on-the-fly. In addition, we show how we can employ our method for the novel task of recovering an estimate for the occluded backside of the imaged objects. Finally, we present results on a variety of object classes and a range of imaging conditions.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

Lifting Object Detection Datasets into 3D (2015, neutral): “Hassner and Basri [27] performed class-based single-view”
Extreme 3D Face Reconstruction: Seeing Through Occlusions (2017, methodology): “These techniques include some of the earliest methods for face reconstruction [15, 16, 53] and more recent approaches [14].”
Depth Transfer: Depth Extraction from Video Using Non-Parametric Sampling (2014, unknown): “Compared to techniques here, we not only focus on depth from a single image , but also present a framework for using temporal information for enhanced and time-coherent depth when multiple frames are available.”
Inferring 3D Shapes and Deformations from Single Views (2010, background): “For the shark data, the shape data set contains eleven 3D shark models of different shark species available from Internet [19].”
Extreme 3D Face Reconstruction: Looking Past Occlusions (2017, methodology): “These include some of the earliest methods for face reconstruction [16, 48] and more recent methods [15].”
Internet Based Morphable Model (2013, background): “Similarly,[24] reconstruct a shape by combining patches from a database of depths, [7] proposed general (non face specific) priors on depth and albedo for shape from shading.”
Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set (2019, methodology): “For example, [20, 19] estimate image depth by building correspondences between the input image and one or a set of reference 3D faces.”
Approximate Nearest Subspace Search with Applications to Pattern Recognition (2007, methodology): “A similar procedure is frequently used in the so called “by-example” patch based methods for applications including segmentation [6] and reconstruction [10].”
Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network (2016, methodology): “Example based methods, beginning from the work of [14] and more recently [13, 39], modify the 3D surface of example face shapes, fitting them to the face appearing in input photo.”
Learning to Maintain Natural Image Statistics (2018, background): “This has been shown effective for a range of applications [17,18,19,20].”
Data-Driven 3D Primitives for Single Image Understanding (2013, methodology): “Non-parametric approaches provide an alternative approach to incorporating global constraints, and instead use patch-to-patch [12], scene [17], or 2D-3D [22] matching to obtain nearest neighbors followed by label transfer.”
Temporal Tessellation: A Unified Approach for Video Analysis (2016, methodology): “estimating per-pixel (dense) correspondences between two images of similar, though not necessarily the same, scenes. One of the earliest methods to show potential applications for this capability was [13]. It was later followed by the SIFT ﬂow of [25], which provided a robust means for computing and”
Maintaining Natural Image Statistics with the Contextual Loss (2018, background): “nt approach is to prefer image-specic priors, learned only from the image to be restored, without any external information from other images. This has been shown eective for a range of applications [17,18,19,20]. Recently, attempts have been made to utilize image-specic priors within the CNN framewo”
Make3D: Learning 3D Scene Structure from a Single Still Image (2009, methodology): “Hassner and Basri [17] used an example-based approac h to estimate depth of an object from a known object class.”
Depth Extraction from Video Using Non-parametric Sampling (2012, background): “Single-image shape from shading is also possible for known (a priori) object classes [10,11].”
Total Moving Face Reconstruction (2014, background): “Most related to our work are single view methods, particularly [30,28,27,29].”