NeRF++: Analyzing and Improving Neural Radiance Fields
By Kai Zhang, Gernot Riegler, Noah Snavely, Vladlen Koltun. 2020
1,237 citations recorded by the source corpus.
Neural Radiance Fields (NeRF) achieve impressive view synthesis results for a variety of capture settings, including 360 capture of bounded scenes and forward-facing capture of bounded and unbounded scenes. NeRF fits multi-layer perceptrons (MLPs) representing view-invariant opacity and view-dependent color volumes to a set of training images, and samples novel views based on volume rendering techniques. In this technical report, we first remark on radiance fields and their potential ambiguities, namely the shape-radiance ambiguity, and analyze NeRF's success in avoiding such ambiguities. Second, we address a parametrization issue involved in applying NeRF to 360 captures of objects within large-scale, unbounded 3D scenes. Our method improves view synthesis fidelity in this challenging scenario. Code is available at this https URL.
Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Generative Models, 3D Vision & Pose
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Graphics and Computer-Aided Design
- Andreas Geiger
- Jonathan T. Barron
- Ben Mildenhall
- Matthias Nießner
- Pratul P. Srinivasan
- Peter Hedman
- Anpei Chen
- Zehao Yu
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Jitendra Malik
- Ming-Hsuan Yang
- Trevor Darrell
- Andreas Geiger
- Xiaogang Wang
- Andrea Vedaldi
In Radiology, Nuclear Medicine and Imaging
- Christian Theobalt
- Jonathan T. Barron
- Andreas Geiger
- Pratul P. Srinivasan
- Matthew Tancik
- Ben Mildenhall
- Jiatao Gu
- Songyou Peng
In Generative Models
- Andreas Geiger
- Eli Shechtman
- Jun-Yan Zhu
- Jonathan T. Barron
- Ben Mildenhall
- Richard Zhang
- Sanja Fidler
- Ming-Yu Liu
In 3D Vision & Pose
- Andreas Geiger
- Matthias Nießner
- Jonathan T. Barron
- Marc Pollefeys
- Songyou Peng
- Michael Niemeyer
- Christian Theobalt
- Noah Snavely
Citations in the corpus
- HexPlane: A Fast Representation for Dynamic Scenes (2023, background): “Also, instead of simply representing everything using spherical coordinates in the paper, we could have a foreground and background model like NeRF++ [92], where the background is modeled in spherical coordinates.”
- Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video (2020, background): “The background needs to be fairly close to the foreground, an issue we “inherit” from NeRF and which could be addressed similarly to NeRF++ [98].”
- MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes (2023, background): “2022], and reconstructing the scene in a warped domain where far-away regions are compressed [Barron et al. 2022; Zhang et al. 2020].”
- Mip-Splatting: Alias-Free 3D Gaussian Splatting (2023, background): “3D radiance field reconstruction from multi-view observations is a well-known ill-posed problem as multiple distinctly different reconstructions can result in the same 2D projections [2, 55, 56].”
- UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction (2021, methodology): “NeRF [34] and follow-ups [6,28,35,36,44,45,49,53,63] use volume rendering by learning alpha-compositing of a radiance field along rays.”
- GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting (2023, positive): “NeRF’s capacity for high-fidelity rendering, coupled with its implicit nature that offers significant expansibility, marks a substantial improvement over conventional approaches[2, 32, 56].”
- 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering (2023, neutral): “3.2.”
- FastNeRF: High-Fidelity Neural Rendering at 200FPS (2021, methodology): “This allows for accelerating existing approaches for: reconstruction of dynamic scenes [30, 34, 14], singleimage reconstruction [39], quality improvement [35, 47] and others [28].”
- Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting (2023, background): “Other works (Zhang et al., 2020; Verbin et al., 2022; Barron et al., 2021; 2022; 2023) aim to enhance rendering quality by addressing existing issues in the vanilla NeRF, such as aliasing and reflection.”
- MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures (2022, background): “For NeRF [33] and NeRF++ [52], we dash entries where the original papers did not report quantitative performance.”
- NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction (2021, methodology): “The outside scene is represented using NeRF++ [53].”
- Zero-Shot Text-Guided Object Generation with Dream Fields (2021, background): “NeRF learns high-frequency and near-field [62] artifacts like partially-transparent “floating“ regions of density.”
- NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, neutral): “Zhang et al. developed the NeRF++ [68] (October 2020) model, which was adapted to generate novel views for unbound scenes, by separating the scene using a sphere.”
- In-Place Scene Labelling and Understanding with Implicit Scene Representation (2021, methodology): “NeRF [16] and other systems based on it [34, 11, 31, 27] use MLPs to overfit input from a single bounded scene and act as an implicit volumetric representation for realistic view-synthesis.”
- NeX: Real-time View Synthesis with Neural Basis Expansion (2021, methodology): “Subsequent work improves upon NeRF by using explicit sparse voxel representation to improve fine detail (NSVF) [18], parameterizing the space to better support unbounded scenes (NeRF++) [50], incorporating learned 2D features that help enforce multiview consistency (GRF) [42], or extending NeRF to h”
- Advances in Neural Rendering (2021, methodology): “NeRF [MST ∗ 20] and related methods (e.g., [MBRS ∗ 21,NG21b, PCPMMN21,SDZ ∗ 21,ZRSK20,NSP ∗ 21]) use differentiable volume rendering to project the scene representations into 2D images.”
- BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering (2021, background): “NeRF has inspired many subsequent works that extend its continuous neural volumetric representation for more practical scenarios beyond simple static scenes, including unbounded scenes [42,3], dynamic scenes [12,41], nonrigidly deforming objects [27,25,26,38], phototourism settings with changing ill”