Neural Importance Sampling

By T. Muller, Brian McWilliams, Fabrice Rousselle, Markus H. Gross, Jan Novák. 2018

435 citations recorded by the source corpus.

We propose to use deep neural networks for generating samples in Monte Carlo integration. Our work is based on non-linear independent components estimation (NICE), which we extend in numerous ways to improve performance and enable its application to integration problems. First, we introduce piecewise-polynomial coupling transforms that greatly increase the modeling power of individual coupling layers. Second, we propose to preprocess the inputs of neural networks using one-blob encoding, which stimulates localization of computation and improves inference. Third, we derive a gradient-descent-based optimization for the Kullback-Leibler and the χ2 divergence for the specific application of Monte Carlo integration with unnormalized stochastic estimates of the target distribution. Our approach enables fast and accurate inference and efficient sample generation independently of the dimensionality of the integration domain. We show its benefits on generating natural images and in two applications to light-transport simulation: first, we demonstrate learning of joint path-sampling densities in the primary sample space and importance sampling of multi-dimensional path prefixes thereof. Second, we use our technique to extract conditional directional densities driven by the product of incident illumination and the BSDF in the rendering equation, and we leverage the densities for path guiding. In all applications, our approach yields on-par or higher performance than competing techniques at equal sample count.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Evaluation & Benchmarks, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

In Probabilistic & Bayesian Methods

Citations in the corpus

Stochastic Normalizing Flows (2020, background): “Importance sampling (M ¨ uller et al., 2018; No ´ e et al., 2019): Generate x k ∼ p X ( x ) and compute observables of interest as 2.”
Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design (2019, background): “M¨uller et al. (2018) introduce piecewise polynomial couplings that are similar in spirit to our mixture of logistics couplings.”
VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, background): “, 2019), light-transport simulation (Müller et al., 2019), molecular modeling (Sanchez-Lengeling & AspuruGuzik, 2018; Noé et al.”
Equivariant Flows: exact likelihood generative learning for symmetric densities (2020, background): “…inference (Rezende & Mohamed, 2015; Tomczak & Welling, 2016; Louizos & Welling, 2017; Berg et al., 2018) or approximate sampling from distributions given by an energy function (Oord et al., 2017), which can be made exact using importance sampling (Müller et al., 2018; Noé et al., 2019).”
Neural Spline Flows (2019, background): “[38] restrict themselves to monotonicallyincreasing linear and quadratic polynomial segments, whose coefficients are parameterized by θi.”
Normalizing Flows for Probabilistic Modeling and Inference (2019, background): “Müller et al. (2019), recognizing the importance of using expressive proposals, employ normalizing flows for q(η).”
Densely connected normalizing flows (2021, methodology): “Still, modeling pθ(x) using bijective formulation enables exact likelihood evaluation and efficient sample generation, which makes this approach convenient for various downstream tasks [21, 22, 23].”
Normalizing Flows: An Introduction and Review of Current Methods (2020, background): “Piecewise-linear and piecewise-quadratic: Müller et al. [2018] used linear splines for coupling functions h : [0, 1]→ [0, 1].”
A RAD approach to deep mixture models (2019, background): “, 2013) or modifications of spline functions (Müller et al., 2019; Durkan et al., 2019; Dolatabadi et al., 2020) that deliberately create discontinuities in the derivative.”
Cubic-Spline Flows (2019, methodology): “r direction; however, more ﬂexible autoregressive transforms have obtained better density-estimation results. More ﬂexible coupling transforms have been proposed, based on piecewise-quadratic splines [20], although these have not previously been evaluated as density estimators. Continuous ﬂows such ”
Instant neural graphics primitives with a multiresolution hash encoding (2022, methodology): “Learning the 2D to RGB mapping of image coordinates to colors has become a popular benchmark for testing a model’s ability to represent high-frequency detail [Martel et al. 2021; Müller et al. 2019; Sitzmann et al. 2020; Tancik et al. 2020].”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, methodology): “Linear and quadratic splines [157], cubic splines [49], and rational-quadratic splines [50] have been applied so far.”
Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM (2023, methodology): “Among these works, NeRF [14] and its variants adopt coordinate encoding [16, 20] with MLPs and show impressive scene reconstruction using differentiable rendering.”
Autoregressive Energy Machines (2019, methodology): “Einstein We generate the Einstein data by sampling co-ordinates proportional to the pixel intensities in an image of Albert Einstein (M¨uller et al., 2018), before adding uniform noise, and re-scaling the resulting points to [0 , 1] 2 .”
Continuous diffusion for categorical data (2022, methodology): “Inspired by Durkan et al. (2019); Müller et al. (2019) we parameterise ˜ 𝐹 ( 𝑡 ) as a monotonic piecewise linear function, which is very straightforward to nor-malise and invert.”
On Embeddings for Numerical Features in Tabular Deep Learning (2022, neutral): “The second one is a generalized version of the one-blob encoding [29] (see subsection E.1 for details).”
TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed (2022, background): “Applications include image generation [15,17], noise modelling [18], video generation [19], audio generation [20–22], graph generation [23], reinforcement learning [24–26], computer graphics [67], and physics [27–31].”