Normalizing Flows: An Introduction and Review of Current Methods

By Ivan Kobyzev, Simon J. D. Prince, Marcus A. Brubaker. 2020

1,546 citations recorded by the source corpus.

Normalizing Flows are generative models which produce tractable distributions where both sampling and density evaluation can be efficient and exact. The goal of this survey article is to give a coherent and comprehensive review of the literature around the construction and use of Normalizing Flows for distribution learning. We aim to provide context and explanation of the models, review current state-of-the-art literature, and identify open questions and promising future directions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Continual Learning via Local Module Composition (2021, methodology): “[48] Ivan Kobyzev, Simon Prince, and Marcus Brubaker.”
StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows (2020, methodology): “We choose Normalizing Flows (NF), as they produce tractable distributions where both sampling and density evaluation is efficient and exact [Kobyzev et al. 2020].”
Generalized Out-of-Distribution Detection: A Survey (2021, neutral): “Flow-based methods (Kobyzev et al., 2020; Zisselman & Tamar, 2020;Kingma&Dhariwal,2018;VanOordetal.,2016;Jiang et al., 2021a) can also be used for probabilistic modeling.”
Bernstein flows for flexible posteriors in variational Bayes (2022, methodology): “The main idea is to enable the VI procedure to approximate the joined posterior of the p model parameters by a flexible variational distribution.”
HuMoR: 3D Human Motion Model for Robust Pose Estimation (2021, background): “Finally, normalizing flows [16] and neural ODEs [8] show exciting potential for modeling human motion, however conditional generation with these models is not yet well-developed.”
Time Series Data Augmentation for Deep Learning: A Survey (2020, methodology): “Another example is normalizing flows (NFs) [Kobyzev et al., 2020], which recently have shown success in modeling time series stochastic processes with excellent inter-/extrapolation performance given observed data [Deng et al.”
InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation (2023, neutral): “Learning one-step model with distillation avoids adversarial training [4, 11, 66] or special invertible neural networks [67, 68, 69].”
TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed (2022, neutral): “Finite Flows Normalizing flows [16, 42–44] use a finite number of transformations to construct differentiable bijections between complex unknown distributions and simple distributions.”
OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport (2020, methodology): “4 Related Works Finite Flows Normalizing flows [31, 42, 44, 50] use a concatenation of discrete transformations, where specific architectures are chosen to allow for efficient inverse and Jacobian determinant computations.”
Stochastic Image-to-Video Synthesis using cINNs (2021, methodology): “Moreover, we parametrize T as an invertible neural network [57, 16, 41] Tθ with parameters θ which, given the image x0, translates between the representations z and ν.”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, background): “A hierarchical VAE with bidirectional inference [125].”
D’ya Like DAGs? A Survey on Structure Learning and Causal Discovery (2021, background): “In CAREFL [126], the authors combine causal discovery with the deep learning framework known as normalizing flows [131, 194].”