Multi-scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation

By Dan Xu, Elisa Ricci, Wanli Ouyang, Xiaogang Wang, Nicu Sebe. Computer Vision and Pattern Recognition, 2017

444 citations recorded by the source corpus.

This paper addresses the problem of depth estimation from a single still image. Inspired by recent works on multi-scale convolutional neural networks (CNN), we propose a deep model which fuses complementary information derived from multiple CNN side outputs. Different from previous methods, the integration is obtained by means of continuous Conditional Random Fields (CRFs). In particular, we propose two different variations, one based on a cascade of multiple CRFs, the other on a unified graphical model. By designing a novel CNN implementation of mean-field updates for continuous CRFs, we show that both proposed models can be regarded as sequential deep networks and that training can be performed end-to-end. Through extensive experimental evaluation we demonstrate the effectiveness of the proposed approach and establish new state of the art results on publicly available datasets.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/03a7d2aa06dd7accfe7d24d08cdcb6af22dfac00

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation (2022, methodology): “Subsequent works introduced some post-processing techniques, such as Conditional Random Fields (CRF), to improve the accuracy [24,25,38].”
Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps With Accurate Object Boundaries (2018, methodology): “…0.954 0.987 Lee et al. [21] 0 Threshold Method P R F1 > 0.25 Eigen and Fergus [8] ∗∗ 0.544 0.481 0.500 Laina et al [16] 0.489 0.435 0.454 Xu et al. [40] 0.516 0.400 0.436 Fu et al. [10] 0.320 0.583 0.402 Ours (SENet-154) 0.644 0.508 0.562 > 0.5 Eigen and Fergus [8] ∗∗ 0.587 0.456 0.501 Laina et al…”
Learning Depth with Convolutional Spatial Propagation Network (2018, background): “ditional random field (CRF) [47], [48], [49], [50], [51], [52] and multi-task correlation with joint training [37], [53], [54], [55].”
High Quality Monocular Depth Estimation via Transfer Learning (2018, methodology): “Monocular depth estimation has been considered by many CNN methods where they formulate the problem as a regression of the depth map from a single RGB image [6, 23, 37, 14, 38, 9].”
Exploiting the Potential of Standard Convolutional Autoencoders for Image Restoration by Evolutionary Search (2018, background): “…is still much room for exploration of search spaces of architectures of classical convolutional networks, which may apply to other tasks such as single image colorization (Zhang et al., 2016), depth estimation (Eigen et al., 2014; Xu et al., 2017), and optical ﬂow estimation (Ilg et al., 2017).”
Learning Deep Structured Multi-Scale Features using Attention-Gated CRFs for Contour Prediction (2017, methodology): “…combination of multi-scale representations has been also widely investigated for other pixel-level prediction tasks, such as semantic segmentation [44], visual saliency detection [21] and monocular depth estimation [40], pedestrian detection [39] and different deep architectures have been designed.”
AdaBins: Depth Estimation Using Adaptive Bins (2020, methodology): “Monocular depth estimation has been considered by many CNN methods as a regression of a dense depth map from a single RGB image [8, 25, 45, 16, 46, 11, 19, 1, 26, 22].”
Joint Task-Recursive Learning for Semantic Segmentation and Depth Estimation (2018, methodology): “The prediction results of the methods in [24, 6] usually have much noise, especially at the object boundaries, curtains, sofa and bed.”
Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimation (2018, methodology): “As demonstrated in our experiments (Section 4), this strategy leads to improved performance with respect to a cascade model as proposed in [36].”
From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation (2019, background): “Xu et al. [49] propose an architecture that exploits multi-scale estimations derived from inner layers by fusing them within a CRF framework.”