Implicit Normalizing Flows
By Cheng Lu, Jianfei Chen, Chongxuan Li, Qiuhao Wang, Jun Zhu. 2021
38 citations recorded by the source corpus.
Normalizing flows define a probability distribution by an explicit invertible transformation $\boldsymbol{\mathbf{z}}=f(\boldsymbol{\mathbf{x}})$. In this work, we present implicit normalizing flows (ImpFlows), which generalize normalizing flows by allowing the mapping to be implicitly defined by the roots of an equation $F(\boldsymbol{\mathbf{z}}, \boldsymbol{\mathbf{x}})= \boldsymbol{\mathbf{0}}$. ImpFlows build on residual flows (ResFlows) with a proper balance between expressiveness and tractability. Through theoretical analysis, we show that the function space of ImpFlow is strictly richer than that of ResFlows. Furthermore, for any ResFlow with a fixed number of blocks, there exists some function that ResFlow has a non-negligible approximation error. However, the function is exactly representable by a single-block ImpFlow. We propose a scalable algorithm to train and draw samples from ImpFlows. Empirically, we evaluate ImpFlow on several classification and density modeling tasks, and ImpFlow outperforms ResFlow with a comparable amount of parameters on all the benchmarks.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
Citations in the corpus
- Fast Lossless Neural Compression with Integer-Only Discrete Flows (2022, background): “…likelihood-based generative models have been developed, including normalizing flows (Dinh et al. (2017); Ho et al. (2019a); Chen et al. (2020b); Lu et al. (2021), NFs), variational auto-encoders (Kingma & Welling (2013); Rezende et al. (2014), VAEs), auto-regressive models (Domke et al.…”