MegaDepth: Learning Single-View Depth Prediction from Internet Photos

By Zhengqi Li, Noah Snavely. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

1,385 citations recorded by the source corpus.

Single-view depth prediction is a fundamental problem in computer vision. Recently, deep learning methods have led to significant progress, but such methods are limited by the available training data. Current datasets based on 3D sensors have key limitations, including indoor-only images (NYU), small numbers of training examples (Make3D), and sparse sampling (KITTI). We propose to use multi-view Internet photo collections, a virtually unlimited data source, to generate training data via modern structure-from-motion and multi-view stereo (MVS) methods, and present a large depth dataset called MegaDepth based on this idea. Data derived from MVS comes with its own challenges, including noise and unreconstructable objects. We address these challenges with new data cleaning methods, as well as automatically augmenting our data with ordinal depth relations generated using semantic segmentation. We validate the use of large amounts of Internet data by showing that models trained on MegaDepth exhibit strong generalization-not only to novel scenes, but also to other diverse datasets including Make3D, KITTI, and DIW, even when no images from those datasets are seen during training.1

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/3395de3126d9b6b9d75f0d85d8c4ebab8ff84686

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

DISK: Learning local features with policy gradient (2020, methodology): “We use a subset of the MegaDepth dataset [19], from which we choose 135 scenes with 63k images in total.”
DIODE: A Dense Indoor and Outdoor DEpth Dataset (2019, background): “Megadepth [20] is a large-scale dataset of outdoor internet images, with depth maps reconstructed using structure-from-motion techniques, but also lacking in ground truth depth and scale.”
Depth-supervised NeRF: Fewer Views and Faster Training for Free (2021, background): “While these tools have become staples of 3D learning, recent works have focused on using deep learning to extract dense depth in the monocular setting [10, 23] or from videos [12].”
Image Matching Across Wide Baselines: From Paper to Practice (2020, methodology): “Megadepth (Li and Snavely 2018) is a more representative, phototourism-based dataset, which using COLMAP, as in our case—it could, in fact, easily be folded into ours.”
BlendedMVS: A Large-Scale Dataset for Generalized Multi-View Stereo Networks (2019, methodology): “As shown in Table 1, R-MVSNet trained on DTU [2] and MegaDepth [15] achieve similar f score performances, while R-MVSNet trained on the proposed dataset outperforms models trained on the other three datasets for all scenes.”
Depth Anything 3: Recovering the Visual Space from Any Views (2025, neutral): “…LiDAR BlendedMVS [115] 503 3D Recon Co3dv2 [68] 30616 Colmap DL3DV [53] 6379 Colmap HyperSim [69] 344 Synthetic MapFree [3] 921 Colmap MegaDepth [51] 268 Colmap MegaSynth [40] 6049 Synthetic MvsSynth [38] 121 Synthetic Objaverse [17] 505557 Synthetic Omniobject [102] 5885 Synthetic OmniWorld…”
Depth Pro: Sharp Monocular Metric Depth in Less Than a Second (2024, positive): “MegaDepth (Li & Snavely, 2018) demonstrated that training on a diverse dataset allows generalizing monocular depth prediction beyond a specific domain.”
Transformers in Remote Sensing: A Survey (2022, methodology): “CF-ViT [165] Image Registration MegaDepth [166] KC A CNN-transformers framework that first performs coarse registration on the down-sampled image, followed by registration of image pairs via a CNN-transformer module with the resulting point pair subsets integrated to obtain final global registration”
A Survey on Deep Learning Techniques for Stereo-Based Depth Estimation (2020, methodology): “For example, Li et al. [28] used modern structure-from-motion and multiview stereo (MVS) methods together with multiview Internet photo collections to create the large-scale MegaDepth dataset providing improved depth estimation accuracy via bigger training dataset sizes.”
Consistent video depth estimation (2020, methodology): “…synthetic [Butler et al. 2012; Mayer et al. 2016a] , specific domains (e.g., driving scenes) [Geiger et al. 2013], single images [Chen et al. 2016; Li et al. 2019; Li and Snavely 2018], or videos (or multiple images) of static scenes [Schops et al. 2017; Silberman et al. 2012; Sturm et al. 2012a].”
MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervision (2024, neutral): “A prevalent approach is to predict relative depth in an affine-invariant manner, either through direct regression [9, 10, 20, 32, 39, 60, 61, 63] or generative modeling [18, 21, 27, 49].”
LoFTR: Detector-Free Local Feature Matching with Transformers (2021, methodology): “We use ScanNet [7] and MegaDepth [21] to demonstrate the effectiveness of LoFTR for pose estimation Category Method Pose estimation AUC”
Learning Target Candidate Association to Keep Track of What Not to Track (2021, background): “Hence, in the domain of sparse keypoint matching, recent learning based approaches leverage large scale datasets [19, 38] such as MegaDepth [36] or ScanNet [9], that provide ground truth matches.”
LightGlue: Local Feature Matching at Light Speed (2023, methodology): “For a more fair evaluation, we perform an extensive outdoor experiment on the test scenes of our MegaDepth [38] split, which covers 4 unique phototourism landmarks that SuperGlue was not trained with: Sagrada Familia, Lincoln Memorial Statue, London Castle, and the British Museum.”
DUSt3R: Geometric 3D Vision Made Easy (2023, positive): “DUSt3R is trained with a mixture of eight datasets: Habitat [104], ARKitScenes [25], MegaDepth [56], Static Scenes 3D [110], Blended MVS [162], Scan-Net++ [166], CO3Dv2 [94] and Waymo [122].”
Single-View View Synthesis With Multiplane Images (2020, methodology): “In Table 3, we compare performance with three depth-prediction methods: MegaDepth [18], Depth in the Wild [29] and the recent “3D Ken Burns Effect” system [22].”