Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields

By Fayao Liu, Chunhua Shen, Guosheng Lin, Ian David Reid. 2015

1,273 citations recorded by the source corpus.

In this article, we tackle the problem of depth estimation from single monocular images. Compared with depth estimation using multiple images such as stereo depth perception, depth from monocular images is much more challenging. Prior work typically focuses on exploiting geometric priors or additional sources of information, most using hand-crafted features. Recently, there is mounting evidence that features from deep convolutional neural networks (CNN) set new records for various vision applications. On the other hand, considering the continuous characteristic of the depth values, depth estimation can be naturally formulated as a continuous conditional random field (CRF) learning problem. Therefore, here we present a deep convolutional neural field model for estimating depths from single monocular images, aiming to jointly explore the capacity of deep CNN and continuous CRF. In particular, we propose a deep structured learning scheme which learns the unary and pairwise potentials of continuous CRF in a unified deep CNN framework. We then further propose an equally effective model based on fully convolutional networks and a novel superpixel pooling method, which is about 10 times faster, to speedup the patch-wise convolutions in the deep model. With this more efficient model, we are able to design deeper networks to pursue better performance. Our proposed method can be used for depth estimation of general scenes with no geometric priors nor any extra information injected. In our case, the integral of the partition function can be calculated in a closed form such that we can exactly solve the log-likelihood maximization. Moreover, solving the inference problem for predicting depths of a test image is highly efficient as closed-form solutions exist. Experiments on both indoor and outdoor scene datasets demonstrate that the proposed method outperforms state-of-the-art depth estimation approaches.

Subfields: Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation (2015, methodology): “The depth estimation method [30] explores super-pixel pooling for building the gap between low-resolution feature map and high-resolution ﬁnal prediction.”
Chained Multi-stream Networks Exploiting Pose, Motion, and Appearance for Action Classification and Detection (2017, methodology): “Encoder-decoder architectures with an up-convolutional part have been used successfully for semantic segmentation tasks [23, 22, 30, 3, 27], depth estimation [20] and optical ﬂow estimation [7].”
3D shape segmentation via shape fully convolutional networks (2017, background): “Later, a great deal of research has 39 emerged based on the FCN algorithms and achieved good re- 40 sults in various fields such as edge detection [21], depth regres- 41 sion [22], optical flow [23], simplifying sketches [24] and so 42 on.”
The limits and potentials of deep learning for robotics (2018, background): “Estimating the depth of the scene from a single image has become a widely researched topic [31], [32], [66].”
(CAD)$^2$RL: Real Single-Image Flight without a Single Real Image (2016, background): “in the hallways, more detailed depth estimation [36, 24, 23] could also be used without loss of generality.”
RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation (2016, methodology): “The depth estimation method [34] employs super-pixel pooling to output high-resolution prediction.”
Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue (2016, methodology): “Errors for other methods are taken from [6,17].”
Maximum Entropy Deep Inverse Reinforcement Learning (2015, background): “…prediction: including pixel-wise semantic segmentation by (Long et al., 2014), sliding window detection and prediction of object boundaries (Sermanet et al., 2013), depth estimation with single monocular images (Liu et al., 2015) and human pose estimation in monocular images (Tompson et al., 2014).”
Pooling Methods in Deep Neural Networks, a Review (2020, neutral): “In [45, 46] a super-pixel pooling is introduced.”
BA-Net: Dense Bundle Adjustment Network (2018, methodology): “Ours Wang et al [8] Zhou et al [56] Godard et al [22] Liu et al [34] Eigen et al [16] ATE(km) 0.”
Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation (2022, methodology): “Subsequent works introduced some post-processing techniques, such as Conditional Random Fields (CRF), to improve the accuracy [24,25,38].”
GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose (2018, methodology): “Our method signiﬁcantly outperforms both supervised methods [9, 28] and previously unsupervised work [14, 56].”
Soft Rasterizer: A Differentiable Renderer for Image-Based 3D Reasoning (2019, methodology): “To simplify the learning problem, recent works reconstruct 3D shape via predicting intermediate 2.5D representations, such as depth map [25], image collections [18], displacement map [16] or normal map [36, 44].”
A Survey on Deep Learning Techniques for Stereo-Based Depth Estimation (2020, methodology): “They have been also explored by Liu et al. [5].”
Monocular depth estimation based on deep learning: An overview (2020, methodology): “[69] to tackle the problem of monocular depth estimation.”