Digging Into Self-Supervised Monocular Depth Estimation

By Clément Godard, Oisin Mac Aodha, Gabriel J. Brostow. IEEE International Conference on Computer Vision, 2018

2,496 citations recorded by the source corpus.

Per-pixel ground-truth depth data is challenging to acquire at scale. To overcome this limitation, self-supervised learning has emerged as a promising alternative for training models to perform monocular depth estimation. In this paper, we propose a set of improvements, which together result in both quantitatively and qualitatively improved depth maps compared to competing self-supervised methods. Research on self-supervised monocular training usually explores increasingly complex architectures, loss functions, and image formation models, all of which have recently helped to close the gap with fully-supervised methods. We show that a surprisingly simple model, and associated design choices, lead to superior predictions. In particular, we propose (i) a minimum reprojection loss, designed to robustly handle occlusions, (ii) a full-resolution multi-scale sampling method that reduces visual artifacts, and (iii) an auto-masking loss to ignore training pixels that violate camera motion assumptions. We demonstrate the effectiveness of each component in isolation, and show high quality, state-of-the-art results on the KITTI benchmark.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/589cfcb2f995c94b0a98c902cc1f5e0f27cbd927

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Is Pseudo-Lidar needed for Monocular 3D Object detection? (2021, methodology): “Leveraging exciting recent progress in depth estimation [11, 13, 15, 16, 31], pseudo-lidar detectors [69, 40, 57] first use a pre-trained depth network to compute an intermediate pointcloud representation, which is then fed to a 3D detection network.”
Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding (2018, background): “This is because when the depth value is estimated large, the car will stay at the same place in the warped image, yielding small photometric errors during training, as also observed in [22].”
Unifying Flow, Stereo and Depth Estimation (2022, methodology): “Monocular methods [65], [66], [67], [68], [69], [70] take a single image as input and use generic network architectures like ResNet [71] to predict the dense depth map, while multi-view methods [12], [20], [37], [72], [73], [74], [75], [76], [77] usually focus on how to encode the geometric…”
Continuous 3D Perception Model with Persistent State (2025, neutral): “Extensive research [7, 27, 42, 55, 58, 59, 85, 92] focuses on estimating monocular depth.”
Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation (2019, methodology): “ro. We now elaborate further on the implementation details for computing the dense correspondences between I1 and z established via (7). As far as the depth map d 1is concerned, we use the Monodepth2 [67] architecture, in particular the model trained with stereo supervision at 1024 320 resolution, t”
SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation (2018, methodology): “To circumvent this behavior, previous methods [6], [8] incorporate a post-processing step that alpha-blends the disparity images from the image and its horizontally flipped version.”
Refine and Distill: Exploiting Cycle-Inconsistency and Knowledge Distillation for Unsupervised Monocular Depth Estimation (2019, methodology): “Other recent works demonstrated that temporal information and, in particular, considering multiple consecutive frames contribute to improve depth estimation [35, 41, 12, 43].”
Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting (2021, positive): “Apart from traditional object classification, detection or semantic segmentation, self-supervision has been extended to sub-domains like human pose-estimation [32], co-part segmentation [27], and depth estimation [19].”
On the Uncertainty of Self-Supervised Monocular Depth Estimation (2020, methodology): “We choose as baseline model Monodepth2 [20], thanks to the code made available and to its possibility to be trained seamlessly according to monocular, stereo, or both self-supervision paradigms.”
BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation (2022, methodology): “Indeed, various strategies have been explored to benefit MDE, such as self-supervised learning [18], multi-task training [38], specific supervision losses [35], sparse ordinal [39] or relative depth estimation [40].”
DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency (2018, background): “, pose [20], depth [36], or flow [57]) can be used for further improving the performance.”
Depth From Videos in the Wild: Unsupervised Monocular Depth Learning From Unknown Cameras (2019, background): “The common 5-point Absolute Trajectory Error (ATE) metric [47, 6, 44, 13] measures local agreement between the estimated trajectories and the respective groundtruth.”
Monocular depth estimation based on deep learning: An overview (2020, methodology): “s also introduced into L vs to quantify the differences between reconstructed and target images: L vs =a 1 SSIM(I n Iˆ n) 2 +(1 a)jI n Iˆ j; (13) where a is a balance factor. Besides, the recent work [81] has proven that it is more efﬁcient to calculate the minimum value of the reconstruction error ”
PixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction (2023, background): “In monocular depth estimation, state-of-the-art models rely on sophisticated scale-invariant depth losses [11, 13, 33, 34].”
Consistent video depth estimation (2020, methodology): “We use the Monodepth2 [Godard et al. 2019] as the base singleimage depth estimation network.”
MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervision (2024, neutral): “3.”
SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving (2023, methodology): “While early methods require full depth annotations to supervise the depth estimation model [15, 25, 66], later research focuses on self-supervised depth estimation as it does not require intensive human annotations [70, 17, 5, 67, 62, 56, 69, 43, 9].”