From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation

By Jin Han Lee, Myung-Kyu Han, Dong Wook Ko, Il Hong Suh. arXiv.org, 2019

812 citations recorded by the source corpus.

Estimating accurate depth from a single image is challenging because it is an ill-posed problem as infinitely many 3D scenes can be projected to the same 2D scene. However, recent works based on deep convolutional neural networks show great progress with plausible results. The convolutional neural networks are generally composed of two parts: an encoder for dense feature extraction and a decoder for predicting the desired depth. In the encoder-decoder schemes, repeated strided convolution and spatial pooling layers lower the spatial resolution of transitional outputs, and several techniques such as skip connections or multi-layer deconvolutional networks are adopted to recover back to the original resolution for effective dense prediction. In this paper, for more effective guidance of densely encoded features to the desired depth prediction, we propose a network architecture that utilizes novel local planar guidance layers located at multiple stages in the decoding phase. We show that the proposed method outperforms the state-of-the-art works with significant margin evaluating on challenging benchmarks. We also provide results from an ablation study to validate the effectiveness of the proposed method.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/4a0e8ee9eece980b54c9899977a3db0af6b8f845

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Neural Window Fully-connected CRFs for Monocular Depth Estimation (2022, methodology): “85 and α is set to 10 following previous works [17].”
Atlas: End-to-End 3D Scene Reconstruction from Posed Images (2020, background): “With the rise of deep learning, monocular depth estimation has seen huge improvements [15,27,28], however their accuracy is still far below state-of-the-art stereo methods.”
ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth (2023, neutral): “[8] 0.878 0.977 0.994 0.111 0.514 0.048 BTS [20] 0.885 0.978 0.994 0.110 0.392 0.047 Yin et al .”
Is Pseudo-Lidar needed for Monocular 3D Object detection? (2021, methodology): “Leveraging exciting recent progress in depth estimation [11, 13, 15, 16, 31], pseudo-lidar detectors [71, 41, 59] ﬁrst use a pre-trained depth network to compute an intermediate pointcloud representation, which is then fed to a 3D detection network.”
BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation (2022, methodology): “Supervised MDE methods [13], [14], [15], [16], [17] apply RGB and ground-truth pairs during the training stage.”
DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries (2021, methodology): “As an alternative to these 2D-based methods, some methods incorporate more 3D computations into our object detection pipeline by applying a 3D reconstruction method like [4, 5, 6] to create a pseudo-LiDAR or range input of the scene from camera images.”
Categorical Depth Distribution Network for Monocular 3D Object Detection (2021, methodology): “Experiments 1, 2 and 3 in Table 4 show the detection performance with separate depth estimation from BTS [27], DORN [15], and CaDDN respectively.”
Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation (2023, positive): “Successive improvements have come from various fronts, including ordinal regression [15], planar guidance maps [24], neural conditional random fields [59], vision transformers [1, 27, 54], a piecewise planarity prior [34], first-order variational constraints [28] and variational autoencoders [32].”
Learning to Upsample by Learning to Sample (2023, background): “Dense prediction refers to a branch of tasks that require point-wise label prediction, such as semantic/instance/panoptic segmentation [2, 39, 40, 8, 7, 13, 11, 16, 19], object detection [33, 4, 24, 36], and monocular depth estimation [38, 18, 3, 21].”
AdaBins: Depth Estimation Using Adaptive Bins (2020, methodology): “[26] and our method on the KITTI dataset.”
SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving (2023, methodology): “While early methods require full depth annotations to supervise the depth estimation model [15, 25, 66], later research focuses on self-supervised depth estimation as it does not require intensive human annotations [70, 17, 5, 67, 62, 56, 69, 43, 9].”
UniDepth: Universal Monocular Metric Depth Estimation (2024, methodology): “In particular, KITTI testing is performed on the corrected Eigen-split test set [13] with the Garg evaluation mask [16], while NYU testing uses the evaluation mask from [24].”