Plenoxels: Radiance Fields without Neural Networks
By Alex Yu, Sara Fridovich-Keil, Matthew Tancik, Qin Chen, B. Recht, Angjoo Kanazawa. Computer Vision and Pattern Recognition, 2021
2,260 citations recorded by the source corpus.
We introduce Plenoxels (plenoptic voxels), a systemfor photorealistic view synthesis. Plenoxels represent a scene as a sparse 3D grid with spherical harmonics. This representation can be optimized from calibrated images via gradient methods and regularization without any neural components. On standard, benchmark tasks, Plenoxels are optimized two orders of magnitude faster than Neural Radiance Fields with no loss in visual quality. For video and code, please see https://alexyu.net/plenoxels.
Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/e91f73aaef155391b5b07e6612f5346dea888f64
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
- Andreas Geiger
- Jonathan T. Barron
- Matthias Nießner
- Pratul P. Srinivasan
- Ben Mildenhall
- Zehao Yu
- Andrea Tagliasacchi
- Anpei Chen
In Computer Graphics and Computer-Aided Design
- Andreas Geiger
- Jonathan T. Barron
- Ben Mildenhall
- Pratul P. Srinivasan
- Matthias Nießner
- Peter Hedman
- Anpei Chen
- Zehao Yu
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Jitendra Malik
- Trevor Darrell
- Andreas Geiger
- Ming-Hsuan Yang
- Xiaogang Wang
- Andrea Vedaldi
Citations in the corpus
- MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes (2023, methodology): “Later, fast radiance field reconstruction was achieved by representing the scene as a grid, stored either densely [Yu et al. 2022], as latent features to be decoded [Karnewar et al. 2022; Liu et al. 2020; Sun et al. 2022], or as latent hash grids [Müller et al. 2022] implemented with specialized…”
- Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation (2022, background): “PERACT is the only agent that achieves > 70% success, whereas all C2FARM-BC versions perform at chance with ∼ 33%, indicating that the global receptive field of the Transformer is crucial for solving the task.”
- MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures (2022, background): “These 3D embeddings can also be used without a trained decoder, for example by directly storing diffuse colors [30] or by encoding view-dependent colors as spherical harmonics [41].”
- Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis (2023, neutral): “Another group trades space for speed by incorporating explicit and localized neural representations [8, 15, 17, 27, 36, 60, 67, 75, 82, 83, 96, 101].”
- NeRF: Representing scenes as neural radiance fields for view synthesis (positive): “This can be extended to higher dimensions using K-planes [ 17 ], which can even operate without any neural layers or MLP (earlier work on plenoxels developed an adaptive octree voxel representation, again without neural components [ 80 ]).”
- Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis (2023, neutral): “Most often, structured space discretizations like voxel grids [10, 24, 28], octrees [6] or hash grids [17] are used to represent the scene.”
- Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM (2023, background): “Given that coordinate encoding-based methods require lengthy training, many follow-up works [7, 10, 27] propose parametric encodings that increase parameter size but speed up the training.”
- Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering (2023, methodology): “For instance, Plenoxel [13] adopted a sparse voxel grid to inter-polate a continuous density field, and represented view-dependent visual effects with Spherical Harmonics.”
- Compact 3D Gaussian Representation for Radiance Field (2023, methodology): “Subsequent research efforts have been directed toward the reduction of the memory footprint while maintaining or even enhancing the performance quality by grid factorization [6, 7, 11, 13, 15, 31], hash grids [30], grid quantization [38, 40] or pruning [10, 35].”
- SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes (2023, background): “Most recently, Gaussian splatting [13] has shown remarkable performance in terms of rendering quality, resolution, and speed.”
- PixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction (2023, background): “However, unlike dense representations like neural fields [29] and voxel grids [12], Gaussian primitives can be rendered via an inexpensive rasterization operation [33].”
- LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS (2023, methodology): “Follow-ups either explore ray re-parameterizations[4, 5], explicit spatial data structures [10, 21, 31, 39, 45, 55, 66], caching and distillation [22, 28, 48, 60], or ray-based representations [2, 54] for speeding up.”
- Fast Dynamic Radiance Fields with Time-Aware Neural Voxels (2022, methodology): “2022] and Plenoxels [Yu et al. 2022], are all targeted at static scenes, unable to model point motions and show bad results.”
- Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly- Throughs (2021, methodology): “We measure traditional NeRF rendering as an additional baseline, which we refer to as Mega-NeRF-Full, and Plenoxels [32] which generates a sparse voxel structure similar to Plenoctree but with trilinear instead of nearest-neighbor interpolation.”
- A Survey on 3D Gaussian Splatting (2024, neutral): “This allows for direct and often faster access to radiance data but at the cost of higher memory usage and potentially lower resolution.”
- 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering (2023, background): “Though some NeRF variants [3, 6, 7, 9, 21, 32, 34] reduce the training time from days to minutes, the rendering process still bears a non-negligible latency.”
- DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation (2023, methodology): “We argue that the occupancy pruning technique used to accelerate NeRF (M ¨ uller et al., 2022; Sara Fridovich-Keil and Alex Yu et al., 2022) is ineffective in generative settings when supervised by the ambiguous SDS loss as opposed to reconstruction settings.”