Dense Depth Priors for Neural Radiance Fields from Sparse Input Views

By Barbara Roessle, Jonathan T. Barron, Ben Mildenhall, Pratul P. Srinivasan, Matthias Nießner. Computer Vision and Pattern Recognition, 2021

460 citations recorded by the source corpus.

Neural radiance fields (NeRF) encode a scene into a neural representation that enables photo-realistic rendering of novel views. However, a successful reconstruction from RGB images requires a large number of input views taken under static conditions — typically up to a few hundred images for room-size scenes. Our method aims to synthesize novel views of whole rooms from an order of magnitude fewer images. To this end, we leverage dense depth priors in order to constrain the NeRF optimization. First, we take advantage of the sparse depth data that is freely available from the structure from motion (SfM) preprocessing step used to estimate camera poses. Second, we use depth completion to convert these sparse points into dense depth maps and uncertainty estimates, which are used to guide NeRF optimization. Our method enables data-efficient novel view synthesis on challenging indoor scenes, using as few as 18 images for an entire scene.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/26b1c7ba30879b54c42eef91ee58fa906d7e26cb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction (2022, background): “Several researchers proposed to incorporate priors such as depth smoothness [43],semantic similarity [29], or sparse MVS point clouds [55] for the task of novel view synthesis from sparse inputs.”
NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior (2022, methodology): “A number of techniques are proposed to improve NeRF’s performance with additional regularisation [13, 26, 54], depth priors [7,32,47,53], surface enhancements [27,43,49] or latent codes [40, 44, 52].”
FreeNeRF: Improving Few-Shot Neural Rendering with Free Frequency Regularization (2023, background): “For instance, external models can be used to acquire normalization-ﬂow regularization [22], perceptual regularization [38], depth supervision [6, 29, 34], and cross-view semantic consistency [11].”
NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, unknown): “Depth/Point Cloud Supervision DS-NeRF (2021)[42] (2021), NerfingMVS (2021) [43] (2021), Urban Radiance Field (2021) [44], PointNeRF (2022) [45]”
Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-Shot Metric Depth and Surface Normal Estimation (2024, unknown): “These unique attributes render both depth and surface normals indispensable in various computer vision applications, including 3D reconstruction [1], [2], [3], neural rendering (NeRF) [4], [5], [6], [7], autonomous driving [8], [9], [10], and robotics [11], [12], [13].”
Compact 3D Gaussian Representation for Radiance Field (2023, background): “…inception, various works have been proposed to enhance performance in diverse scenarios, such as different resolutions of reconstruction [2, 3], the reduced number of training samples [32, 36, 42, 44, 47], and reconstruction of large realistic scenes [29, 41] and dynamic scenes [12, 23, 24, 33].”
Gen3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control (2025, unknown): “Several works proposed to address these issues through regularization using geometric priors [10, 38, 43, 48, 54, 63, 70, 75], which, however, are sensitive to noise in the estimated depth or normals.”
Decomposing NeRF for Editing via Feature Field Distillation (2022, methodology): “Pre-trained vision models have been used for regularizing NeRF via flows [62], multi-view consistency [35], perceptual loss [110], or depth estimation [100, 77].”
DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization (2024, background): “The first group [11, 31] is to extract accurate but sparse depth values from reliable point clouds, and the second [14, 36, 40, 42, 55] distills depth knowledge from current powerful monocular depth estimators [29, 30].”
ReconFusion: 3D Reconstruction with Diffusion Priors (2023, unknown): “DDP-NeRF [41] further uses a CNN to obtain dense depth supervision from sparse inputs.”