Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimation

By Dan Xu, Wei Wang, Hao Tang, Hong Liu, Nicu Sebe, Elisa Ricci. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

321 citations recorded by the source corpus.

Recent works have shown the benefit of integrating Conditional Random Fields (CRFs) models into deep architectures for improving pixel-level prediction tasks. Following this line of research, in this paper we introduce a novel approach for monocular depth estimation. Similarly to previous works, our method employs a continuous CRF to fuse multi-scale information derived from different layers of a front-end Convolutional Neural Network (CNN). Differently from past works, our approach benefits from a structured attention model which automatically regulates the amount of information transferred between corresponding features at different scales. Importantly, the proposed attention model is seamlessly integrated into the CRF, allowing end-to-end training of the entire architecture. Our extensive experimental evaluation demonstrates the effectiveness of the proposed method which is competitive with previous methods on the KITTI benchmark and outperforms the state of the art on the NYU Depth V2 dataset.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/1231e9ea7fe18e8d6cc7fd0b0285c3644b5e9bed

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Neural Window Fully-connected CRFs for Monocular Depth Estimation (2022, methodology): “To better combine CRFs and networks, other methods integrate CRFs into the layers of the neural networks and train the whole framework endto-end [11, 20, 29, 38].”
Monocular depth estimation based on deep learning: An overview (2020, background): “is proposed in [72] to strengthen the information transfer between corresponding features.”
High Quality Monocular Depth Estimation via Transfer Learning (2018, methodology): “Monocular depth estimation has been considered by many CNN methods where they formulate the problem as a regression of the depth map from a single RGB image [7, 23, 37, 14, 38, 9].”
PAD-Net: Multi-tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing (2018, background): “The other works mainly focused on multi-scale feature learning and ensembling [5, 52, 16], end-to-end structure prediction with CRF models [34, 1, 60, 55] and designing convolutional encoderdecoder network structures [41, 2].”
AdaBins: Depth Estimation Using Adaptive Bins (2020, methodology): “Monocular depth estimation has been considered by many CNN methods as a regression of a dense depth map from a single RGB image [8, 25, 45, 16, 46, 11, 19, 1, 26, 22].”
Refine and Distill: Exploiting Cycle-Inconsistency and Knowledge Distillation for Unsupervised Monocular Depth Estimation (2019, methodology): “[39, 38] exploited CRFs within a deep architecture in order to fuse information at multiple scales.”
On the Uncertainty of Self-Supervised Monocular Depth Estimation (2020, result): “The advent of deep learning, together with the increasing availability of ground truth depth data, led to the development of frameworks [38, 40, 74, 15] achieving unpaired accuracy compared to previous approaches [58, 37, 14].”
Learning Depth with Convolutional Spatial Propagation Network (2018, background): “[51] also propose to learn attention guided multi-scale CRF for depth estimation, which shares similar spirits with ours.”
Pattern-Affinitive Propagation Across Depth, Surface Normal and Semantic Segmentation (2019, background): “6, compared with the recent work [60], our predictions are more fine-detailed and closer to ground truth.”