IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression

By Rianne van den Berg, Alexey A. Gritsenko, Mostafa Dehghani, Casper Kaae Sønderby, Tim Salimans. 2020

68 citations recorded by the source corpus.

In this paper we analyse and improve integer discrete flows for lossless compression. Integer discrete flows are a recently proposed class of models that learn invertible transformations for integer-valued random variables. Due to its discrete nature, they can be combined in a straightforward manner with entropy coding schemes for lossless compression without the need for bits-back coding. We discuss the potential difference in flexibility between invertible flows for discrete random variables and flows for continuous random variables and show that (integer) discrete flows are more flexible than previously claimed. We furthermore investigate the influence of quantization operators on optimization and gradient bias in integer discrete flows. Finally, we introduce modifications to the architecture to improve the performance of this model class for lossless compression.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Fast Lossless Neural Compression with Integer-Only Discrete Flows (2022, methodology): “Different from IDF (Hoogeboom et al., 2019; van den Berg et al., 2020) which model x , z both with discrete integers, Ho et al. (2019b) dicretize continuous variables and propose a local bits back (LBB) scheme for compression with a general class of NFs which model x , z as continuous.”
Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions (2021, background): “In addition ﬂows for ordinal discrete data (integers) have been explored in (Hoogeboom et al., 2019; van den Berg et al., 2020).”
iFlow: Numerically Invertible Flows for Efficient Lossless Compression via a Uniform Coder (2021, methodology): “To resolve this issue, Integer Discrete Flows (IDF) [18, 4] (Fig.”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, background): “Nevertheless, this can be avoided by embedding the data into a space with more values than the data, making IDFs more flexible than discrete flows [12].”
iVPF: Numerical Invertible Volume Preserving Flow for Efficient Lossless Compression (2021, methodology): “Like general lossless compression algorithms [15, 3, 36, 13], we use CIFAR10, ImageNet32 and ImageNet64 for training and evaluation.”