Instant neural graphics primitives with a multiresolution hash encoding
By T. Muller, Alex Evans, Christoph Schied, Alexander Keller. 2022
6,052 citations recorded by the source corpus.
Neural graphics primitives, parameterized by fully connected neural networks, can be costly to train and evaluate. We reduce this cost with a versatile new input encoding that permits the use of a smaller network without sacrificing quality, thus significantly reducing the number of floating point and memory access operations: a small neural network is augmented by a multiresolution hash table of trainable feature vectors whose values are optimized through stochastic gradient descent. The multiresolution structure allows the network to disambiguate hash collisions, making for a simple architecture that is trivial to parallelize on modern GPUs. We leverage this parallelism by implementing the whole system using fully-fused CUDA kernels with a focus on minimizing wasted bandwidth and compute operations. We achieve a combined speedup of several orders of magnitude, enabling training of high-quality neural graphics primitives in a matter of seconds, and rendering in tens of milliseconds at a resolution of 1920×1080.
Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
- Andreas Geiger
- Matthias Nießner
- Jonathan T. Barron
- Andrea Tagliasacchi
- Pratul P. Srinivasan
- Ben Mildenhall
- Michael Niemeyer
- Leonidas J. Guibas
In Computer Graphics and Computer-Aided Design
- Andreas Geiger
- Jonathan T. Barron
- Matthias Nießner
- Ben Mildenhall
- Pratul P. Srinivasan
- Zehao Yu
- Anpei Chen
- Vincent Lepetit
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Dacheng Tao
- Jitendra Malik
- Trevor Darrell
- Xiaogang Wang
- Andrea Vedaldi
- Andreas Geiger
Citations in the corpus
- Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis (2023, neutral): “Most often, structured space discretizations like voxel grids [10, 24, 28], octrees [6] or hash grids [17] are used to represent the scene.”
- NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, neutral): “03 600k+150k+1000k >12h ∼2000 Instant-NGP (2022) [48] Hash [48] Occupancy [48] 33.”
- Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM (2023, methodology): “Recent efforts focus on sparse alternatives to these parametric embeddings such as octrees [28], tri-plane [2], hash-grid [15] or sparse voxel grid [12, 13] to improve the memory efficiency of dense grids.”
- FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting (2023, methodology): “Muller et al. [31] adopt a multiresolution hash encoding technique that reduces the training time to 5 seconds.”
- PixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction (2023, background): “Discrete data structures can accelerate rendering [6, 12, 25, 30] but fall short of real-time rendering at high resolutions.”
- MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction (2022, methodology): “…the interpolated features at each level and concatenate them together: As the total number of grid cells grows cubically, we use a fixed number of parameters to store the feature grids and use a spatial hash function to index the feature vector at finer levels [45] (see supplementary for details).”
- LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS (2023, methodology): “1, including highly efficient voxel-based NeRFs (Plenoxel[67], Instant-NGP [45]), the compact MLP-based Mip-NeRF360 [5], NeRF with vector quantization [38], and 3D Gaussian Splatting [32].”
- Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis (2023, background): “On this benchmark we compare to Particle-NeRF [1], InstantNGP [26], and TiNeuVox [9].”
- DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision (2023, neutral): “Using these NVS methods to generalize to un-seen real-world scenarios requires training on a large set of real-world multi-view images.”
- Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors (2023, methodology): “We leverage Instant-NGP [57] as our NeRF implementation because of its fast inference and ability to recover complex geometry.”
- LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation (2024, methodology): “We first train an efficient NeRF [34] using the rendered images from 3D Gaussians on-the-fly, and then convert the NeRF to polygonal meshes [48].”
- Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis (2023, neutral): “Another group trades space for speed by incorporating explicit and localized neural representations [8, 15, 17, 27, 36, 60, 67, 75, 82, 83, 96, 101].”
- Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction (2022, background): “In the future, it will be interesting to explore accelerating neural implicit surfaces learning by volume rendering through super-fast per-scene radiance field optimization methods [22, 32].”
- SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion (2024, methodology): “In the coarse stage, we train an Instant-NGP [32] NeRF representation to reconstruct the SV3D-generated images ( i.e . without SDS loss) at a lower resolution.”
- Fast Dynamic Radiance Fields with Time-Aware Neural Voxels (2022, methodology): “regularization techniques for complicated real scenes, integrating geometric/motion priors for domain-specific scenes, and further introducing compressing/pruning techniques like voxel hashing [Müller et al. 2022; Nießner et al. 2013] and tensor decomposition [Chen et al.”
- MVDream: Multi-view Diffusion for 3D Generation (2023, methodology): “multi-resolution hash-grid (Müller et al., 2022), from threestudio as its first stage base representation (except TextMesh which uses SDF).”