Lossy Image Compression with Compressive Autoencoders

By Lucas Theis, Wenzhe Shi, Andrew Cunningham, Ferenc Huszár. 2017

1,156 citations recorded by the source corpus.

We propose a new approach to the problem of optimizing autoencoders for lossy image compression. New media formats, changing hardware technology, as well as diverse requirements and content types create a need for compression algorithms which are more flexible than existing codecs. Autoencoders have the potential to address this need, but are difficult to optimize directly due to the inherent non-differentiabilty of the compression loss. We here show that minimal changes to the loss are sufficient to train deep autoencoders competitive with JPEG 2000 and outperforming recently proposed approaches based on RNNs. Our network is furthermore computationally efficient thanks to a sub-pixel architecture, which makes it suitable for high-resolution images. This is in contrast to previous work on autoencoders for compression using coarser approximations, shallower architectures, computationally expensive methods, or focusing on small images.

Subfields: Computer Vision and Pattern Recognition, Generative Models, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

In Efficiency & Compression

Citations in the corpus

The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization (2020, methodology): “For our experiments, we specifically use the CAE [31] and EDSR [20] architectures as the basis for DeepAugment.”
Neural Discrete Representation Learning (2017, methodology): “[36] use scalar quantisation to compress activations for lossy image compression before arithmetic encoding.”
Evaluating Lossy Compression Rates of Deep Generative Models (2020, background): “Other researchers have developed practical lossy coding schemes achieving variational rate distortion bounds for particular latent variable models which exploited the factorial structure of the variational posterior (Ballé et al., 2018; Theis et al., 2017; Yang et al., 2020).”
Predictive Sampling with Forecasting Autoregressive Models (2020, methodology): “Setting In this section we explore autoencoders with a probabilistic latent space (Theis et al., 2017; Ball ´ e et al., 2017).”
High-Fidelity Generative Image Compression (2020, methodology): “, the straight-through estimator as in [44]), which ensures that G sees the same quantization noise during training and inference.”
Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations (2017, methodology): “For the architecture, we use a variant of the convolutional autoencoder proposed recently in [30] (see Appendix A.”
Low Bit-rate Speech Coding with VQ-VAE and a WaveNet Decoder (2019, background): “These models are able to learn the redundancies in signals directly by being exposed to many examples during training, and have been successfully applied in the domain of image compression [1]–[4].”
Learned Video Compression (2018, background): “In the last two years, we have seen a great surge of ML-based image compression approaches [13, 39, 40, 5, 4, 12, 20, 38, 33, 18, 2, 22, 6, 3, 8, 27, 28].”
LOGAN: Membership Inference Attacks Against Generative Models (2017, background): “ng distribution of a given training dataset. In particular, generative models offer a way to artiﬁcially generate plausible images and videos and they are used in many applications, e.g., compression [60], denoising [7], inpainting [69], superresolution [35], semi-supervised learning [54], etc. In t”
LOGAN: Evaluating Privacy Leakage of Generative Models Using Generative Adversarial Networks (2017, background): “They are used in a plethora of applications, such as compression [40], denoising [4], inpainting [45], super-resolution [25], semisupervised learning [36], clustering [39], and deep neural networks pre-training [15] in cases where labeled data is expensive.”
Structured Disentangled Representations (2018, background): “, 2017], and lossy image compression [Theis et al., 2017].”
Real-Time Adaptive Image Compression (2017, methodology): “Theis et al. (2016) and Ball ´ e et al. (2016) quantize rather than binarize, and propose strategies to approximate the entropy of the quantized representation: this provides them with a proxy to penalize it.”
Generative Adversarial Networks for Extreme Learned Image Compression (2018, background): “Common loss functions to measure the distortion between the original and decompressed images are the mean-squared error (MSE) (Theis et al., 2017; Ball´e et al., 2016b; Agustsson et al., 2017; Li et al., 2017; Ball´e et al., 2018; Torfason et al., 2018), or perceptual metrics such as MS-SSIM…”
Pros and Cons of GAN Evaluation Measures (2018, background): “For example, the likelihood is good for measuring compression methods [79] while psychophysics and user ratings are fit for evaluating image reconstruction and synthesis methods [45, 29].”
Video Compression through Image Interpolation (2018, background): “For images, deep networks yield state-of-the-art compression ratios with impressive reconstruction quality [6,12,22,25,26].”
Generative Compression (2017, background): “Noteworthy examples include the compressive autoencoder [9], which derives differentiable approximations for quantization and entropy estimation to allow training by backpropagation.”
Channel-Wise Autoregressive Entropy Models for Learned Image Compression (2020, methodology): “Most recent research in learned image compression uses deep neural networks, and a wide range of model architectures have been explored including recurrent networks [1]–[4] and autoencoders with an entropy-constrained bottleneck [5]– [16].”
Piano Genie (2018, background): “compression literature that also use autoencoders with discrete bottlenecks [2, 26].”
Improving Inference for Neural Image Compression (2020, methodology): “[M7] CAE Pioneering “Compressive Autoencoder” model proposed in [Theis et al., 2017].”
NASH: Toward End-to-End Neural Architecture for Generative Semantic Hashing (2018, methodology): “…qθ(x|z)] dz dσ(giφ(x)) dφ (5) Although this is clearly a biased estimator, it has been shown to be a fast and efficient method relative to other gradient estimators for discrete variables, especially for the Bernoulli case (Bengio et al., 2013; Hubara et al., 2016; Theis et al., 2017).”