Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields
By Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, Peter Hedman. Computer Vision and Pattern Recognition, 2021
2,737 citations recorded by the source corpus.
Though neural radiance fields (NeRF) have demon-strated impressive view synthesis results on objects and small bounded regions of space, they struggle on “un-bounded” scenes, where the camera may point in any di-rection and content may exist at any distance. In this set-ting, existing NeRF-like models often produce blurry or low-resolution renderings (due to the unbalanced detail and scale of nearby and distant objects), are slow to train, and may exhibit artifacts due to the inherent ambiguity of the task of reconstructing a large scene from a small set of images. We present an extension of mip-NeRF (a NeRF variant that addresses sampling and aliasing) that uses a non-linear scene parameterization, online distillation, and a novel distortion-based regularizer to overcome the chal-lenges presented by unbounded scenes. Our model, which we dub “mip-NeRF 360” as we target scenes in which the camera rotates 360 degrees around a point, reduces mean-squared error by 57% compared to mip-NeRF, and is able to produce realistic synthesized views and detailed depth maps for highly intricate, unbounded real-world scenes.
Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/ec90ffa017a2cc6a51342509ce42b81b478aefb3
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
- Andreas Geiger
- Jonathan T. Barron
- Matthias Nießner
- Ben Mildenhall
- Pratul P. Srinivasan
- Vincent Lepetit
- Antoine Guédon
- Zehao Yu
In Computer Graphics and Computer-Aided Design
- Andreas Geiger
- Jonathan T. Barron
- Ben Mildenhall
- Pratul P. Srinivasan
- Matthias Nießner
- Anpei Chen
- Vincent Lepetit
- Antoine Guédon
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Dacheng Tao
- Xiaogang Wang
- Jitendra Malik
- Trevor Darrell
- Chen Change Loy
- Andrea Vedaldi
Citations in the corpus
- Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis (2023, positive): “Experiments were performed with Mip-Nerf360 [ 2] dataset. cients and shape parameters for clustering.”
- VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction (2024, positive): “To address this issue, we follow Mip-NeRF 360 [3] to perform color correction on the rendered images before evaluating the metrics of all the methods, which solves a per-image least squares problem to align the RGB values between the rendered image and its corresponding ground truth.”
- gsplat: An Open-Source Library for Gaussian Splatting (2024, neutral): “We compare the training performance and efficiency of gsplat training against the original implementation by Kerbl et al. on the MipNeRF360 dataset (Barron et al., 2022).”
- Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction (2023, background): “The hybrid approaches confer the dual benefits of expediting both training and inference phases while producing outcomes on par with advanced frameworks [2, 3].”
- SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis (2023, background): “Neural radiance fields (NeRFs) [27, 23, 1, 2, 35] have made tremendous progress in generating photo-realistic novel views of scenes by optimizing implicit function representations given a set of 2D input views.”
- DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes (2023, neutral): “Our method achieves the optimal quality with less rendering time compared to those of NeRF-based methods designed for unbounded scenes(e.g., Mip-NeRF [1], Mip-NeRF360 [2], Urban-NeRF [12]).”
- DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization (2024, background): “The following improvements mainly pursue either higher quality [2, 3] or efficiency [6, 12, 15, 21, 26, 37, 53], but hard to achieve both.”
- Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting (2023, background): “Recent breakthroughs, such as NeRF (Mildenhall et al., 2020), have achieved photorealistic static scene rendering (Barron et al., 2021; Verbin et al., 2022).”
- PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics (2023, background): “Building upon the achievements of NeRF, further studies have focused on enhancing reconstruction quality and improving training speeds [1, 7, 20, 26, 40, 46].”
- HexPlane: A Fast Representation for Dynamic Scenes (2023, background): “NeRF [46] and its variants [2, 3, 44, 48, 74, 76, 85, 92] show impressive results on novel view synthesis [9,80,87,99] and many other applications including 3D reconstruction [42, 72, 90, 94, 100], semantic segmentation [27,59,98], generative model [5,6,10, 49,62,82], and 3D content creation…”
- Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering (2023, methodology): “We found this is conceptually similar to the proposal network utilized in [4], serving as the geometry proxy estimator for efficient sampling.”
- LERF: Language Embedded Radiance Fields (2023, background): “The scale s represents the side length in world co-ordinates of a cube centered at (cid:126)x , and is analogous to how Mip-NeRF[3, 4] incorporates different scales via integrated positional encodings.”
- BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering (2021, background): “Despite early attempts on representing multiscale 3D scenes have been made in [2,3,14], it remains unclear how well a neural radiance field can handle scenarios at drastically varied scales.”