Deep Generative Models for Distribution-Preserving Lossy Compression

By Michael Tschannen, Eirikur Agustsson, Mario Lucic. 2018

155 citations recorded by the source corpus.

We propose and study the problem of distribution-preserving lossy compression. Motivated by recent advances in extreme image compression which allow to maintain artifact-free reconstructions even at very low bitrates, we propose to optimize the rate-distortion tradeoff under the constraint that the reconstructed samples follow the distribution of the training data. The resulting compression system recovers both ends of the spectrum: On one hand, at zero bitrate it learns a generative model of the data, and at high enough bitrates it achieves perfect reconstruction. Furthermore, for intermediate bitrates it smoothly interpolates between learning a generative model of the training data and perfectly reconstructing the training samples. We study several methods to approximately solve the proposed optimization problem, including a novel combination of Wasserstein GAN and Wasserstein Autoencoder, and present an extensive theoretical and empirical characterization of the proposed compression systems.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

High-Fidelity Generative Image Compression (2020, methodology): “Furthermore, adversarial losses were previously employed to improve the visual quality of neural compression systems [39, 40, 48, 3, 9].”
The Perception-Distortion Tradeoff (2017, background): “GAN-based image restoration Most recently, GAN-based methods have demonstrated unprecedented perceptual quality in super-resolution [3], [6], [9], [37], inpainting [7], [20], [38], compression [21], [39], [40], deblurring [41] and image-to-image translation [11], [22], [42].”
The GAN Landscape: Losses, Architectures, Regularization, and Normalization (2018, background): “The power of these models was recently harnessed in a variety of applications, including image generation, learned compression, and domain transfer (Isola et al., 2017; Radford et al., 2016; Agustsson et al., 2018; Tschannen et al., 2018).”
Recent Advances in Autoencoder-Based Representation Learning (2018, background): “Speciﬁcally, Larsen et al. [82], Rosca et al. [83] combine VAE with standard GAN [34], and Tschannen et al. [84] equip AAE/WAE with a Wasserstein GAN loss [85].”
On Self Modulation for Generative Adversarial Networks (2018, background): “, 2017), learned compression (Tschannen et al., 2018), super-resolution (Ledig et al.”
Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff (2019, methodology): “This was done on an image patch level (Rippel & Bourdev, 2017), on reduced-size (thumbnail) images (Tschannen et al., 2018; Santurkar et al., 2018), on a full-image scale (Agustsson et al.”
Image-to-Image Translation: Methods and Applications (2021, background): “in making predictions [42], estimating missing data [43], compressing datasets [44] and generating invisible data.”
Practical Lossless Compression with Latent Variables using Bits Back Coding (2019, methodology): “Ball´e et al. (2017), Theis et al. (2017), Ball´e et al. (2018), and Minnen et al. (2018) all implement lossy compression using (variational) auto-encoder style models, and Tschannen et al. (2018) train a model for lossy compression using a GAN-like objec-tive.”
A Large-Scale Study on Regularization and Normalization in GANs (2018, unknown): “These models were recently applied to great effect in a variety of applications, including image generation, learned compression, and domain adaptation (Brock et al., 2019; Menick & Kalchbrenner, 2019; Karras et al., 2019; Lucic et al., 2019; Isola et al., 2017; Tschannen et al., 2018).”
Practical Full Resolution Learned Lossless Image Compression (2018, background): “However, recent work on learned compression using deep neural networks has solely focused on lossy compression [4, 41, 42, 34, 1, 3, 44].”
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications (2022, unknown): “This idea has been identified as the distribution-preserving image compression in [61].”
Finite Scalar Quantization: VQ-VAE Made Simple (2023, background): “Other compression work relied on bounding the range of the quantizer (Mentzer et al., 2018; Tschannen et al., 2018; Agustsson et al., 2019).”
MineGAN: Effective Knowledge Transfer From GANs to Target Domains With Few Images (2019, background): “They have been shown to generate high-quality realistic images [14, 15, 4] and are used in many applications including image manipulation [13, 38], style transfer [9], compression [31], and colorization [36].”