ReconFusion: 3D Reconstruction with Diffusion Priors

By Rundi Wu, Ben Mildenhall, Philipp Henzler, Keunhong Park, Ruiqi Gao, Daniel Watson, Pratul P. Srinivasan, Dor Verbin, Jonathan T. Barron, Ben Poole and 1 more. Computer Vision and Pattern Recognition, 2023

374 citations recorded by the source corpus.

3D reconstruction methods such as Neural Radiance Fields (NeRFs) excel at rendering photorealistic novel views of complex scenes. However, recovering a high-quality NeRF typically requires tens to hundreds of input images, resulting in a time-consuming capture process. We present ReconFusion to reconstruct real-world scenes using only a few photos. Our approach leverages a diffusion prior for novel view synthesis, trained on synthetic and multiview datasets, which regularizes a NeRF-based 3D reconstruction pipeline at novel camera poses beyond those captured by the set of input images. Our method synthesizes realistic geometry and texture in underconstrained regions while preserving the appearance of observed regions. We perform an extensive evaluation across various real-world datasets, including forward-facing and 360-degree scenes, demonstrating significant performance improvements over previous few-view NeRF reconstruction approaches. Please see our project page at reconfusion.github. io.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/114e83828084ed6e82b3984979b2ec8f6e7f9cf7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images (2024, background): “…., Scene Representation Networks (SRN) [34], Neural Radiance Fields (NeRF) [24] and Light Filed Networks (LFN) [33], these methods are still not satisfactory for practical applications due to expensive per-scene optimization [26,39,43], high memory cost [3,18,44] and slow rendering speed [40,49].”
FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting (2023, neutral): “While these NeRF-like models present strong performance on various benchmarks, they generally require several hours of training time.”
NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, neutral): “However, by using Contrastive Language Image Pre-training (CLIP), CLIP-NeRF could extract from user input text or images, the induced latent space displacements by using shape and appearance mapper networks.”
Gen3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control (2025, positive): “With the recent success of image/video generation models, Re-conFusion [61] and CAT3D [12] started leveraging prior knowledge learned by these models to facilitate sparse-view NVS. Due to the necessity of per-scene optimization, these methods remain inherently slow.”
Continuous 3D Perception Model with Persistent State (2025, neutral): “With rapid advances in image and video generative models [57, 61, 62], much of the current research in 3D generation focuses on transferring priors from images and videos to 3D [45, 48, 56, 91].”
Recent advances in 3D Gaussian splatting (2024, neutral): “Apart from the works mentioned above, other methods focus on special input data including images without camera [83, 84], blurry inputs [85], CT scans [86], panoramic images [87], and satellite images [88].”
ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis (2024, neutral): “Reconfusion [41] proposes a PixelNeRF [10] Fig.”