3-D Depth Reconstruction from a Single Still Image

By Ashutosh Saxena, Sung H. Chung, Andrew Y. Ng. 2007

679 citations recorded by the source corpus.

Abstract We consider the task of 3-d depth estimation from a single still image. We take a supervised learning approach to this problem, in which we begin by collecting a training set of monocular images (of unstructured indoor and outdoor environments which include forests, sidewalks, trees, buildings, etc.) and their corresponding ground-truth depthmaps. Then, we apply supervised learning to predict the value of the depthmap as a function of the image. Depth estimation is a challenging problem, since local features alone are insufficient to estimate depth at a point, and one needs to consider the global context of the image. Our model uses a hierarchical, multiscale Markov Random Field (MRF) that incorporates multiscale local- and global-image features, and models the depths and the relation between depths at different points in the image. We show that, even on unstructured scenes, our algorithm is frequently able to recover fairly accurate depthmaps. We further propose a model that incorporates both monocular cues and stereo (triangulation) cues, to obtain significantly more accurate depth estimates than is possible using either monocular or stereo cues alone.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Lifting Object Detection Datasets into 3D (2015, positive): “To account for differences between the 3D exemplars and the object depicted, Su and Guibas [26] choose a few exemplars that appear to be similar to the depicted shape and combine them to produce a single depth map.”
PixelNet: Towards a General Pixel-level Architecture (2016, background): “These include low-level tasks such as edge detection [16, 50, 71] and optical ﬂow [3, 18], mid-level tasks such as depth/normal recovery [4, 19, 20, 60, 68], and high-level tasks such as keypoint prediction [28, 58], object detection [33], and semantic segmentation [13, 22, 31, 48, 52, 61].”
Inferring 3D Shapes and Deformations from Single Views (2010, background): “[18] have investigated to recover rough depth estimate from image features.”
Discrete-Continuous Depth Estimation from a Single Image (2014, methodology): “In contrast, several methods have been proposed to directly estimate the depth of image (super)pixels [24, 25].”
Beat the MTurkers: Automatic Image Labeling from Weak 3D Supervision (2014, background): “Exact inference can be performed in both cases [26].”
Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images (2013, background): “Some works have addressed the task of inferring coarse 3D layout of outdoor scenes, exploiting appearance and geometric information [11, 25].”
Indoor Scene Understanding with RGB-D Images: Bottom-up Segmentation, Object Detection and Semantic Segmentation (2015, background): “Some works have addressed the task of inferring coarse 3D layout of outdoor scenes, exploiting appearance and geometric information [22,43].”
Soft Rasterizer: Differentiable Rendering for Unsupervised Single-View Mesh Reconstruction (2019, background): “5D representations, such as depth map [8, 39, 42, 7, 24] or normal map [1, 50, 34, 51].”
D Traffic Scene Understanding from Movable Platforms (2013, background): “on estimating 3D from single images [36], [39], [50], [54], [55].”
Second-order constrained parametric proposals and sequential search-based structured prediction for semantic segmentation in RGB-D images (2015, background): “The scientific problem of three dimensional scene understanding from images, both quantitative[49, 12] and qualitative[21, 41], has a long standing research tradition in computer vision.”
Joint Semantic Segmentation and 3D Reconstruction from Monocular Video (2014, background): “Semantic segmentation can be used to estimate 3D information [22,10,25].”
Single-Image Depth Perception in the Wild (2016, background): “Depth from a Single Image: Image-to-depth is a long-standing problem with a large body of literature [19, 20, 12, 1, 6, 7, 8, 9, 10, 19, 21, 22, 23, 24, 25, 26].”
Recovering Occlusion Boundaries from an Image (2011, background): “[62, 63] train with range images to estimate depth from the image features directly.”
Toward Holistic Scene Understanding: Feedback Enabled Cascaded Classification Models (2012, methodology): “The state-of-the-art classifiers improve on the base model by explicitly hand designing the task specific probabilistic model [46], [59] or by using ad hoc methods to implicitly use information from other tasks [33].”
Autonomous MAV flight in indoor environments using single image perspective cues (2011, background): “They compute image features that capture single image distance cues (Make3D, [5], [24], [25]) to predict distances to obstacles.”
Depth and surface normal estimation from monocular images using regression on deep features and hierarchical CRFs (2015, methodology): “These work generally uses hand crafted features [10, 12].”