Generative Compression

By Shibani Santurkar, David M. Budden, Nir Shavit. 2017

200 citations recorded by the source corpus.

Traditional image and video compression algorithms rely on hand-crafted encoder/decoder pairs (codecs) that lack adaptability and are agnostic to the data being compressed. We describe the concept of generative compression, the compression of data using generative models, and suggest that it is a direction worth pursuing to produce more accurate and visually pleasing reconstructions at deeper compression levels for both image and video data. We also show that generative compression is orders- of-magnitude more robust to bit errors (e.g., from noisy channels) than traditional variable-length coding schemes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

The Perception-Distortion Tradeoff (2017, background): “While the two tradeoffs are different, it is important to note that perceptual quality does play a role in lossy compression, as evident from the success of recent GAN based compression schemes [39], [40], [54].”
Generative Adversarial Networks for Extreme Learned Image Compression (2018, methodology): “[36] use a GAN framework to learn a generative model over thumbnail images, which is then used as a decoder for thumbnail image compression.”
Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff (2019, methodology): “For example, incorporating generative adversarial type losses has been shown to lead to signiﬁcantly better perceptual quality, but at the cost of increased distortion (Tschan-nen et al., 2018; Agustsson et al., 2018; Santurkar et al., 2018).”
ClusterGAN : Latent Space Clustering in Generative Adversarial Networks (2018, background): “Generative compression is yet another application involving Z [26].”
Learned Image Compression With Discretized Gaussian Mixture Likelihoods and Attention Modules (2020, background): “Some approaches [12, 13, 14] use generative models to learn the distribution of images using adversarial training to achieve better subjective quality at extremely low bit rate.”
Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks (2017, background): “Recently, there has been a surge in research applying neural networks to the problem of image compression [21, 22, 23, 10, 4, 20, 2, 19].”
Toward Semantic Communications: Deep Learning-Based Image Semantic Coding (2022, background): “However, it is not feasible to directly adopt the naive GANs for image semantic decoding, since the generated image tend to deviate significantly from the input image in finegrained features [2], [3], [39].”
Variational image compression with a scale hyperprior (2018, methodology): “Santurkar et al. (2017) formulate their compression method in a hybrid VAE-GAN framework, adopting a stepwise training scheme where a decoder is first trained using an adversarial loss. It is then fixed, and an encoder is trained to minimize the reconstruction error. Rippel and Bourdev (2017) also e”
Context-adaptive Entropy Model for End-to-end Optimized Image Compression (2018, background): “First, as a consequence of the recent success of generative models, some image compression approaches targeting the superior perceptual quality (Agustsson et al., 2018; Santurkar et al., 2018; Rippel & Bourdev, 2017) have been proposed.”
Optimizing the Latent Space of Generative Networks (2017, background): “Santurkar et al. [2017] also use generative models to compress images.”
Video Compression With Rate-Distortion Autoencoders (2019, methodology): “The models are typically trained to minimize the mean squared error between original and decompressed images or by using more perceptual metrics such as MSSSIM [32] or adversarial loss [34].”
Deep Generative Models for Distribution-Preserving Lossy Compression (2018, methodology): “[34] use a generator trained with a GAN as a decoder in a compression system.”