Block Neural Autoregressive Flow

By Nicola De Cao, Ivan Titov, Wilker Aziz. 2019

142 citations recorded by the source corpus.

Normalising flows (NFS) map two density functions via a differentiable bijection whose Jacobian determinant can be computed efficiently. Recently, as an alternative to hand-crafted bijections, Huang et al. (2018) proposed neural autoregressive flow (NAF) which is a universal approximator for density functions. Their flow is a neural network (NN) whose parameters are predicted by another NN. The latter grows quadratically with the size of the former and thus an efficient technique for parametrization is needed. We propose block neural autoregressive flow (B-NAF), a much more compact universal approximator of density functions, where we model a bijection directly using a single feed-forward network. Invertibility is ensured by carefully designing each affine transformation with block matrices that make the flow autoregressive and (strictly) monotone. We compare B-NAF to NAF and other established flows on density estimation and approximate inference for latent variable models. Our proposed flow is competitive across datasets while using orders of magnitude fewer parameters.

Subfields: Artificial Intelligence, Generative Models, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Generative Models

In Model Architectures

Citations in the corpus

Stochastic Normalizing Flows (2020, background): “…2018; M ¨ uller et al., 2018), which are a subclass of autoregressive ﬂows (Germain et al., 2015; Papamakarios et al., 2017; Huang et al., 2018; De Cao et al., 2019; Durkan et al., 2019), and (ii) residual ﬂows (Chen et al., 2018; Zhang et al., 2018; Grathwohl et al., 2018; Behrmann et al.,…”
Equivariant Flows: exact likelihood generative learning for symmetric densities (2020, background): “…2018; Müller et al., 2018), which are a subclass of autoregressive flows (Germain et al., 2015; Papamakarios et al., 2017; Huang et al., 2018; De Cao et al., 2019; Durkan et al., 2019), and 2) residual flows (Chen et al., 2018; Zhang et al., 2018; Grathwohl et al., 2018; Behrmann et al., 2018;…”
Normalizing Flows for Probabilistic Modeling and Inference (2019, methodology): “Examples of flow-based models that use masking include IAF (Kingma et al., 2016), MAF (Papamakarios et al., 2017), NAF (Huang et al., 2018), B-NAF (De Cao et al., 2019), masked invertible network (MintNet) (Song et al., 2019) and masked convolutional generative flow (MaCow) (Ma et al., 2019).”
Automatic Guide Generation for Stan via NumPyro (2021, unknown): “• AutoBNAFNormal (Block Neural Autoregressive Flow [Cao et al. 2019]), • AutoDiagonalNormal (Stan’s mean-field), • AutoMultivariateNormal (Stan’s full-rank), • AutoIAFNormal (Inverse Autoregressive Flow [Kingma et al. 2016]), • AutoLaplaceApproximation (quadratic approximation), •…”
The Power Spherical distribution (2020, background): “Though neural autoregressive ﬂows (Huang et al., 2018; De Cao et al., 2019) have been shown to be remarkably ﬂexible, they are still subject to topological constraints (Dinh et al., 2019; Cor-nish et al., 2019), which motivates extensions for complex manifolds (Brehmer & Cranmer, 2020; Wu et al.,…”
Unconstrained Monotonic Neural Networks (2019, methodology): “Examples include Neural Autoregressive Flows [NAF, Huang et al., 2018] and Block Neural Autoregressive Flows [B-NAF, De Cao et al., 2019].”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, background): “namely Neural Autoregressive Flows (NAF) [95] and Block NAF [38].”
Why Normalizing Flows Fail to Detect Out-of-Distribution Data (2020, background): “We hypothesize that many of our conclusions about coupling layers extend at a high level to other types of normalizing ﬂows [e.g., 3, 6, 12, 20, 13, 30, 38, 17, 8].”