AutoInt: Automatic Integration for Fast Neural Volume Rendering

By David B. Lindell, Julien N. P. Martel, Gordon Wetzstein. Computer Vision and Pattern Recognition, 2020

297 citations recorded by the source corpus.

Numerical integration is a foundational technique in scientific computing and is at the core of many computer vision applications. Among these applications, neural volume rendering has recently been proposed as a new paradigm for view synthesis, achieving photorealistic image quality. However, a fundamental obstacle to making these methods practical is the extreme computational and memory requirements caused by the required volume integrations along the rendered rays during training and inference. Millions of rays, each requiring hundreds of forward passes through a neural network are needed to approximate those integrations with Monte Carlo sampling. Here, we propose automatic integration, a new framework for learning efficient, closed-form solutions to integrals using coordinate-based neural networks. For training, we instantiate the computational graph corresponding to the derivative of the coordinate-based network. The graph is fitted to the signal to integrate. After optimization, we reassemble the graph to obtain a network that represents the antiderivative. By the fundamental theorem of calculus, this enables the calculation of any definite integral in two evaluations of the network. Applying this approach to neural rendering, we improve a tradeoff between rendering speed and image quality: improving render times by greater than 10× with a tradeoff of reduced image quality.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/4fc6e9545d30539fac6a6a0dcac1418ad4dd9a9f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

PlenOctrees for Real-time Rendering of Neural Radiance Fields (2021, methodology): “AutoInt [22] modiﬁes the architecture of the NeRF so that inference requires fewer samples but produces lower quality results.”
Advances in Neural Rendering (2021, background): “AutoInt [LMW21] trains a network to “automatically integrate” the output color value along ray segments by supervising the gradient of the network to behave like a standard NeRF MLP.”
NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, unknown): “AutoInt (Dec 2020) [47] approximates the volume rendering step.”
Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction (2021, methodology): “Several follow-up methods [15,18,29,30,42,43,66] have shown significant speedup of FPS in testing phase, some of which even achieve real-time rendering.”
FastNeRF: High-Fidelity Neural Rendering at 200FPS (2021, methodology): “Orthogonal to this, AutoInt [15] showed that a neural network can be used to approximate the integrals along each ray with far fewer samples.”
Plenoxels: Radiance Fields without Neural Networks (2021, methodology): “AutoInt [20] restructures the coordinatebased MLP to compute ray integrals exactly, for more than 10× faster rendering with a small loss in quality.”
3D Neural Field Generation Using Triplane Diffusion (2022, unknown): “…learn geometry [1,4,5,10,11,15,21,23,45, 47,48,57,65,73] or use posed images to jointly optimize geometry and appearance [6,7,18,25,29,33,36–39, 44, 49, 50, 54, 55 Triplanesareefficient, scalingwiththesurfacearearatherthan volume, and naturally integrate with expressive, fine-tuned 2D generator…”
SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis (2023, unknown): “proaches [47, 38, 22, 43, 15, 32, 16, 28] attempted to speed up the standard NeRF.”
GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds (2021, background): “Follow-up works have tried to improve the output quality [31, 68], make it faster to train and evaluate [30, 31, 41, 51, 59], extend it to deformable objects [15, 28, 48, 50, 63], account for lighting [9, 6, 37, 57] and compositionality [17, 44, 46, 67], as well as add generative capabilities [11, 5”
Fast Dynamic Radiance Fields with Time-Aware Neural Voxels (2022, methodology): “Some works propose to reduce inference times for acceleration by improving sampling strategies [Arandjelović and Zisserman 2021; Fang et al. 2021; Lindell et al. 2021; Neff et al. 2021; Piala and Clark 2021] or introducing efficient rendering techniques [Sitzmann et al. 2021].”
KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs (2021, methodology): “AutoInt replaces the need for numerical integration in NeRF by learning a closed form solution of the antiderivative [23].”