Estimating Depth From Monocular Images as Classification Using Deep Fully Convolutional Residual Networks

By Yuanzhouhan Cao, Zifeng Wu, Chunhua Shen. IEEE transactions on circuits and systems for video technology (Print), 2016

454 citations recorded by the source corpus.

Depth estimation from single monocular images is a key component in scene understanding. Most existing algorithms formulate depth estimation as a regression problem due to the continuous property of depths. However, the depth value of input data can hardly be regressed exactly to the ground-truth value. In this paper, we propose to formulate depth estimation as a pixelwise classification task. Specifically, we first discretize the continuous ground-truth depths into several bins and label the bins according to their depth ranges. Then, we solve the depth estimation problem as classification by training a fully convolutional deep residual network. Compared with estimating the exact depth of a single point, it is easier to estimate its depth range. More importantly, by performing depth classification instead of regression, we can easily obtain the confidence of a depth prediction in the form of probability distribution. With this confidence, we can apply an information gain loss to make use of the predictions that are close to ground-truth during training, as well as fully-connected conditional random fields for post-processing to further improve the performance. We test our proposed method on both indoor and outdoor benchmark RGB-Depth datasets and achieve state-of-the-art performance.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/d27c6800894441054685ffdf896fab15cebf5fd6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimation (2018, methodology): “In [2] a deep residual network is also employed but the problem of depth estimation from still images is translated from a regression to a classification task.”
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment (2023, methodology): “…to multiple (N) modalities, and our dataset VIDAL-10M benefits more downstream tasks beyond VL tasks, including video retrieval (Luo et al., 2022), depth classification (Cao et al., 2017), infrared classification (Baffa & Lattari, 2018) and audio classification (Palanisamy et al., 2020).”
Real-Time Monocular Depth Estimation Using Synthetic Data with Domain Adaptation via Image Style Transfer (2018, background): “Since then, numerous improvements have been made to achieve better directly supervised training for monocular depth estimation [43, 80, 40, 8].”
Enforcing Geometric Constraints of Virtual Normal for Depth Prediction (2019, methodology): “In particular we follow [4] to use the weighted cross-entropy loss (WCEL), with the weight being the information gain.”
Unsupervised Monocular Depth Estimation with Left-Right Consistency (2016, methodology): “Several works have built upon the success of this approach using techniques such as CRFs to improve accuracy [35], changing the loss from regression to classification [5], using other more robust loss functions [33], and incorporating strong scene priors in the case of the related problem of surface”
From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation (2019, background): “Several works follow the success of this approach by incorporating strong scene priors for surface normal estimation [44], using conditional random fields to improve accuracy [27, 23, 40] or changing the learning problem from regression to classification [5].”
Parse geometry from a line: Monocular depth estimation with partial laser observation (2016, methodology): “[9] regarded the depth estimation as a classification task, where the estimation probability could be obtained for refinement using CRF.”
Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps With Accurate Object Boundaries (2018, methodology): “The methods denoted without a superscript [9, 16, 40, 25, 2, 3, 24, 21, 32, 10] and ours should be able to be compared in an equal condition.”
Depth From Videos in the Wild: Unsupervised Monocular Depth Learning From Unknown Cameras (2019, background): “Recently, learning-based approaches for dense depth prediction have gained attention [9, 22, 19, 45].”