Enforcing Geometric Constraints of Virtual Normal for Depth Prediction

By Wei Yin, Yifan Liu, Chunhua Shen, Youliang Yan. 2019

494 citations recorded by the source corpus.

Monocular depth prediction plays a crucial role in understanding 3D scene geometry. Although recent methods have achieved impressive progress in evaluation metrics such as the pixel-wise relative error, most methods neglect the geometric constraints in the 3D space. In this work, we show the importance of the high-order 3D geometric constraints for depth prediction. By designing a loss term that enforces one simple type of geometric constraints, namely, virtual normal directions determined by randomly sampled three points in the reconstructed 3D space, we can considerably improve the depth prediction accuracy. Furthermore, we can not only predict accurate depth but also achieve high-quality other 3D information from the depth without retraining new parameters, Significantly, the byproduct of this predicted depth being sufficiently accurate is that we are now able to recover good 3D structures of the scene such as the point cloud and surface normal directly from the depth, eliminating the necessity of training new sub-models as was previously done. Experiments on two challenging benchmarks: NYU Depth-V2 and KITTI demonstrate the effectiveness of our method and state-of-the-art performance.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

FCOS: Fully Convolutional One-Stage Object Detection (2019, background): “Recently, fully convolutional networks (FCNs) [20] have achieved tremendous success in dense prediction tasks such as semantic segmentation [20, 28, 9, 19], depth estimation [17, 31], keypoint detection [3] and counting [2].”
Conditional Convolutions for Instance Segmentation (2020, background): “FCNs also have shown excellent performance on many other per-pixel prediction tasks ranging from low-level image processing such as denoising, super-resolution; to mid-level tasks such as optical flow estimation and contour detection; and high-level tasks including recent single-shot object detectio”
Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-Shot Metric Depth and Surface Normal Estimation (2024, unknown): “Supervised monocular depth estimation: Following the establishment of several benchmarks [58], [59], neural network-based methods [17], [19], [30] have dominated this task.”
Depth Anything V2 (2024, unknown): “Other works [94, 96, 89] focus on the data-driven perspective.”
AdaBins: Depth Estimation Using Adaptive Bins (2020, methodology): “For comparison, we also used the same strategy for competing methods for which pretrained models are available [26, 47, 4] and report results in Table.”
From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation (2019, methodology): “However, there is some accuracy gap when compared to the current best supervised approach [51].”
GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image (2024, methodology): “Some existing approaches propose to explicitly acquire the surface normal from the depth map by using some geometric constraints, such as Sobel-like operator [19, 27], differentiable least square [36, 44], or randomly sampled point triplets [37, 69, 70].”
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data (2024, background): “Following this, many works consistently improve the depth estimation accuracy by carefully designing the regression task as a classification task [3, 34], introducing more priors [32, 53, 75, 82], and better objective functions [67, 77], etc .”
Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image (2023, background): “As depth distribution corresponding to different RGBs can vary to a large extent, some methods [66, 4] discretize the depth and formulate this problem to a classification [64], which often achieves better performance.”
Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video (2019, background): “However, although these supervised methods [5, 6, 18–21] show high-quality flow and depth estimation results, it is expensive to acquire ground truth in real-world scenes.”
ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth (2023, unknown): “[48] ResNeXt-101 114M BTS [20] DenseNet-161 47M AdaBins [5] EfﬁcientNet-B5 78M LocalBins [6] EfﬁcientNet-B5 74M NeWCRFs [50] Swin-L 270M ZoeDepth (S-L) Beit-L 345M Table 17.”
FCOS: A Simple and Strong Anchor-Free Object Detector (2020, background): “mantic segmentation [10], [11], [12], depth estimation [13], [14], keypoint detection [15] and counting.”