DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision

By Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu and 10 more. Computer Vision and Pattern Recognition, 2023

496 citations recorded by the source corpus.

We have witnessed significant progress in deep learning-based 3D vision, ranging from neural radiance field (NeRF) based 3D representation learning to applications in novel view synthesis (NVS). However, existing scene-level datasets for deep learning-based 3D vision, limited to ei-ther synthetic environments or a narrow selection of real-world scenes, are quite insufficient. This insufficiency not only hinders a comprehensive benchmark of existing methods but also caps what could be explored in deep learning-based 3D analysis. To address this critical gap, we present DL3DV-10K, a large-scale scene dataset, featuring 51.2 million frames from 10,510 videos captured from 65 types of point- of-interest (POI) locations, covering both bounded and unbounded scenes, with different levels of reflection, transparency, and lighting. We conducted a comprehensive benchmark of recent NVS methods on DL3DV-10K, which revealed valuable insights for future research in NVS. In addition, we have obtained encouraging results in a pilot study to learn generalizable NeRF from DL3DV-10K, which manifests the necessity of a large-scale scene-level dataset to forge a path toward a foundation model for learning 3D representation. Our DL3DV-10K dataset, benchmark results, and models will be publicly accessible.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/ac980fe6382a89d50ebf395f626b8677541a2547

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Ying Shan
2. Jinbo Xing
3. Xiaoyu Li
4. Wangbo Yu
5. Li Yuan
6. Xiangjun Gao
7. Wenbo Hu
8. Tien-Tsin Wong

Citations in the corpus

• Continuous 3D Perception Model with Persistent State (2025, unknown): “Examples of our datasets include CO3Dv2 [60], ARK-itScenes [5], ScanNet++ [95], TartanAir [87], Waymo [75], MegaDepth [41], MapFree [3], DL3DV [44], and Dynam-icStereo [35].”
• Gen3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control (2025, unknown): “We select three real-world videos datasets: RE10K [74], DL3DV [29], Waymo Open Dataset (WOD) [49], and a synthetic dataset Kubric4D [14, 53].”
• Depth Anything 3: Recovering the Visual Space from Any Views (2025, unknown): “For training the NVS model, we leverage the large-scale DL3DV dataset [53], which provides diverse real-world scenes with camera poses estimated by COLMAP.”
• VGGT: Visual Geometry Grounded Transformer (2025, unknown): “Hence, while the feed-forward mode of VGGT outperforms all previous alternatives (whether they are feed-forward or not), there is still room for improvement since post-optimization still brings benefits.”
• ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis (2024, methodology): “Our model was trained on a mixed dataset consisting of DL3DV [70] and RealEstate10K [7].”