Real-Time Adaptive Image Compression

By Oren Rippel, Lubomir D. Bourdev. 2017

595 citations recorded by the source corpus.

We present a machine learning-based approach to lossy image compression which outperforms all existing codecs, while running in real-time. Our algorithm typically produces files 2.5 times smaller than JPEG and JPEG 2000, 2 times smaller than WebP, and 1.7 times smaller than BPG on datasets of generic images across all quality levels. At the same time, our codec is designed to be lightweight and deployable: for example, it can encode or decode the Kodak dataset in around 10ms per image on GPU. Our architecture is an autoencoder featuring pyramidal analysis, an adaptive coding module, and regularization of the expected codelength. We also supplement our approach with adversarial training specialized towards use in a compression setting: this enables us to produce visually pleasing reconstructions for very low bitrates.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Fixing the train-test resolution discrepancy (2019, background): “Convolutional Neural Networks [21] (CNNs) are used extensively in computer vision tasks such as image classification [20], object detection [30], inpainting [41], style transfer [11] and even image compression [31].”
High-Fidelity Generative Image Compression (2020, methodology): “[39] uses an adversarial loss as a component in their full-resolution compression system, but they do not systematically ablate and asses the benefits of this loss on the quality of their reconstructions.”
Low Bit-rate Speech Coding with VQ-VAE and a WaveNet Decoder (2019, background): “These models are able to learn the redundancies in signals directly by being exposed to many examples during training, and have been successfully applied in the domain of image compression [1]–[4].”
The Perception-Distortion Tradeoff (2017, background): “Specifically, motivated by [20, 31, 46, 34, 32, 14], restoration problems can be approached by modifying the loss of the generator of a GAN to be”
Learned Video Compression (2018, methodology): “Each video features a separate R-D curve computed from all available compression rates for a given codec: as a number of papers [5, 38] discuss in detail, different ways of summarizing these R-D curves can lead to very different results.”
Generative Adversarial Networks for Extreme Learned Image Compression (2018, methodology): “The work of [34] trains and evaluates a deep compression system optimized for the classical MS-SSIM [45] metric.”
Generative Compression (2017, methodology): “Also noteworthy is the autoencoder setup with pyramidal analysis, adaptive coding and inclusion of a patch-based adversarial loss presented in [19].”
Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff (2019, methodology): “This was done on an image patch level (Rippel & Bourdev, 2017), as a post-processing step (Galteri et al., 2017), and also on a full-image scale (Tschannen et al., 2018; Agustsson et al., 2018; Santurkar et al., 2018).”
Learned Image Compression With Discretized Gaussian Mixture Likelihoods and Attention Modules (2020, background): “Some approaches [12, 13, 14] use generative models to learn the distribution of images using adversarial training to achieve better subjective quality at extremely low bit rate.”
DVC: An End-To-End Deep Video Compression Framework (2018, methodology): “Recently, deep neural network (DNN) based autoencoder for image compression [34, 11, 35, 8, 12, 19, 33, 21, 28, 9] has achieved comparable or even better performance than the traditional image codecs like JPEG [37], JPEG2000 [29] or BPG [1].”
Global Guarantees for Enforcing Deep Generative Priors by Empirical Risk (2017, methodology): “Deep generative models have also empirically been used directly for compression [47].”
Conditional Probability Models for Deep Image Compression (2018, methodology): “MS-SSIM reportedly correlates better with human perception of distortion than mean squared error (MSE).”
Autoencoder Based Image Compression: Can the Learning be Quantization Independent? (2018, unknown): “In the best image compression algorithms based on autoencoders [5, 6, 7], one transform is learned per ratedistortion point at a given quantization step size.”
Adversarial Transfer Learning (2018, methodology): “[103, 104] used a reconstruction and adversarial loss with an autoencoder for learning higher quality image compression at low bit rates.”
Video Compression With Rate-Distortion Autoencoders (2019, methodology): “The models are typically trained to minimize the mean squared error between original and decompressed images or by using more perceptual metrics such as MSSSIM [32] or adversarial loss [34].”
Variational image compression with a scale hyperprior (2018, methodology): “In terms of MS-SSIM, the hyperprior model consistently outperforms conventional codecs as well as Rippel and Bourdev (2017), the current state-of-the-art model for that metric.”
Variable Rate Deep Image Compression With a Conditional Autoencoder (2019, methodology): “[4] (optimized for MS-SSIM) Rippel & Bourdev [19] Mentzer et al.”
Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks (2017, methodology): “This figure shows MS-SSIM (dB) on Kodak: (a) our method compared to [21], [19] and [23] (without entropy coding), and (b) compared to standard image compression codecs.”
A Survey of Unsupervised Deep Domain Adaptation (2018, methodology): “[195, 196] used a reconstruction and adversarial loss with an autoencoder for learning higher quality image compression at low bit rates.”