Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
By Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, Pratul P. Srinivasan. 2021
2,835 citations recorded by the source corpus.
The rendering procedure used by neural radiance fields (NeRF) samples a scene with a single ray per pixel and may therefore produce renderings that are excessively blurred or aliased when training or testing images observe scene content at different resolutions. The straightforward solution of supersampling by rendering with multiple rays per pixel is impractical for NeRF, because rendering each ray requires querying a multilayer perceptron hundreds of times. Our solution, which we call "mip-NeRF" (à la "mipmap"), extends NeRF to represent the scene at a continuously-valued scale. By efficiently rendering anti-aliased conical frustums instead of rays, mip-NeRF reduces objectionable aliasing artifacts and significantly improves NeRF’s ability to represent fine details, while also being 7% faster than NeRF and half the size. Compared to NeRF, mip-NeRF reduces average error rates by 17% on the dataset presented with NeRF and by 60% on a challenging multiscale variant of that dataset that we present. Mip-NeRF is also able to match the accuracy of a brute-force supersampled NeRF on our multiscale dataset while being 22× faster.
Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Graphics and Computer-Aided Design
- Andreas Geiger
- Jonathan T. Barron
- Ben Mildenhall
- Pratul P. Srinivasan
- Matthias Nießner
- Matthew Tancik
- Vincent Sitzmann
- Andrea Tagliasacchi
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Jitendra Malik
- Trevor Darrell
- Andrea Vedaldi
- Andrew Zisserman
In Generative Models
- Richard Zhang
- Eli Shechtman
- Jun-Yan Zhu
- Daniel Cohen-Or
- Andreas Geiger
- Robin Rombach
- Alexei A. Efros
- Stefano Ermon
Citations in the corpus
- On Buggy Resizing Libraries and Surprising Subtleties in FID Calculation (2021, background): “While it was replaced by operations such as max-pooling, based on empirical performance [65], recent works have demonstrated that antialiasing can be compatible and improve performance in convolutional networks [79,85], transformers [57], NeRFs [4], and GANs [36].”
- Urban Radiance Fields (2021, methodology): “Mip-NeRF [4] proposed a scale-aware scene representation based on conical frustrums instead of rays to compensate for blurring and aliasing artifacts.”
- Text-to-3D using Gaussian Splatting (2023, neutral): “After works have emerged to improve NeRF in reconstruction quality [7, 8, 76], handling large-scale [13, 40, 68, 81] and dynamic scenes [3, 47, 51, 60, 73], improving training [10, 44, 67, 79], rendering [25, 56, 80] speed and facilitating down-stream tasks [17, 46, 78, 83, 84].”
- Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction (2023, methodology): “In terms of quality improvement, mipNeRF [2] pi-oneered an effective anti-aliasing method, which was later incorporated into the grid-based approach by zipNeRF [4].”
- PGSR: Planar-Based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction (2024, positive): “We compare with current state-of-the-art meth-ods for pure novel view synthesis as well as similar reconstruction methods to ours, including NeRF [11], Deep Blend-ing [67], INGP [17], [15], NeuS [19], [20], SuGaR [21], 2DGS [55], and GOF [56].”
- SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes (2023, methodology): “Recent advancements in this area are primarily driven by neural radiance fields (NeRF) [19, 30, 37, 66], which utilizes an implicit func-1 tion to simultaneously learn scene geometry [26, 29] and textures [12, 57] from multi-view images.”
- Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields (2021, methodology): “With this small optimization, off-axis IPE features are only modestly more expensive to compute than the axis-aligned IPE features used in mip-NeRF.”
- DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes (2023, neutral): “Our method achieves the optimal quality with less rendering time compared to those of NeRF-based methods designed for unbounded scenes(e.g., Mip-NeRF [1], Mip-NeRF360 [2], Urban-NeRF [12]).”
- DreamFusion: Text-to-3D using 2D Diffusion (2022, methodology): “In mip-NeRF, this is replaced by an integrated positional encoding that accounts for the “width” of the ray being rendered (based on its pixel footprint in the image plane) and the length of each interval [di, di+1] sampled along the ray (Barron et al., 2021).”
- gsplat: An Open-Source Library for Gaussian Splatting (2024, neutral): “Viewing a 3D scene represented by Gaussians at varying resolutions can cause aliasing effects, as seen in prior 3D representations (Barron et al., 2021, 2022).”
- StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation (2021, background): “, hair), which are recently addressed in Alias-free GAN [5] or MipNeRF [1].”
- Mip-Splatting: Alias-Free 3D Gaussian Splatting (2023, methodology): “Multi-scale Training and Multi-scale Testing: Following previous work [1, 17], we train our model with multi-scale data and evaluate on multi-scale data.”
- NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images (2021, methodology): “Given MLP outputs c i , σ i for respective ray segments [ t i 1 , t i ) with lengths ∆ i (see [3]), these weights are If we define a piecewise-constant probability distribution p w over the ray segments using these weights, then our variance regularizer is equal to Calculating the mean (expected…”
- Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation (2023, methodology): “renders 2D images x = g(θ) from a modified Mip-NeRF [4] parameterized as θ.”