End-to-end Optimized Image Compression

By Johannes Ballé, Valero Laparra, Eero P. Simoncelli. International Conference on Learning Representations, 2016

2,121 citations recorded by the source corpus.

We describe an image compression method, consisting of a nonlinear analysis transformation, a uniform quantizer, and a nonlinear synthesis transformation. The transforms are constructed in three successive stages of convolutional linear filters and nonlinear activation functions. Unlike most convolutional neural networks, the joint nonlinearity is chosen to implement a form of local gain control, inspired by those used to model biological neurons. Using a variant of stochastic gradient descent, we jointly optimize the entire model for rate-distortion performance over a database of training images, introducing a continuous proxy for the discontinuous loss function arising from the quantizer. Under certain conditions, the relaxed loss function may be interpreted as the log likelihood of a generative model, as implemented by a variational autoencoder. Unlike these models, however, the compression model must operate at any given point along the rate-distortion curve, as specified by a trade-off parameter. Across an independent set of test images, we find that the optimized method generally exhibits better rate-distortion performance than the standard JPEG and JPEG 2000 compression methods. More importantly, we observe a dramatic improvement in visual quality for all images at all bit rates, which is supported by objective quality estimates using MS-SSIM.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://arxiv.org/abs/1611.01704

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Predictive Sampling with Forecasting Autoregressive Models (2020, methodology): “Setting In this section we explore autoencoders with a probabilistic latent space (Theis et al., 2017; Ball ´ e et al., 2017).”
High-Fidelity Generative Image Compression (2020, methodology): “At the same time, deep learning-based lossy compression has seen great interest [45, 5, 31], where a neural network is directly optimized for the rate-distortion trade-off, which led to new state-of-the-art methods.”
Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations (2017, methodology): “The work of [5] build a deep model using multiple generalized divisive normalization (GDN) layers and their inverses (IGDN), which are specialized layers designed to capture local joint statistics of natural images.”
Differentiable Model Compression via Pseudo Quantization Noise (2021, methodology): “Additive noise was also studied in the context of image compression (Ballé et al., 2017; Choi et al., 2019) in order to provide a differentiable pseudo-quantization operator. However, those work rely on an explicit estimation of the quantized values entropy, in particular with respect to a distribut”
Learned Video Compression (2018, methodology): “Each video features a separate R-D curve computed from all available compression rates for a given codec: as a number of papers [5, 33] discuss in detail, different ways of summarizing these R-D curves can lead to very different results.”
Barlow Twins: Self-Supervised Learning via Redundancy Reduction (2021, background): “, 2018; Schwartz & Simoncelli, 2001) for recent efforts), and has led to a number of algorithms for supervised and unsupervised learning (Ballé et al., 2017; Deco & Parra, 1997; Földiák, 1990; Linsker, 1988; Redlich, 1993a;b; Schmidhuber et al., 1996).”
Demystifying Parallel and Distributed Deep Learning (2018, methodology): “optimize the network to return the original sample (e.g., using a squared loss function), in order to learn de-noising filters. Alternatively, similar techniques can be used to learn image compression[14]. GANs[81] are a recent development in machine learning. They employ deep neural networks to gen”
Real-Time Adaptive Image Compression (2017, methodology): “(4)Reconstructions by Ballé et al. (2016) of images in the Kodak dataset can be found at http://www.”
Generative Adversarial Networks for Extreme Learned Image Compression (2018, methodology): “To investigate the effect of the GAN term in our total loss, we train a baseline model with an MSE loss only (with the same architecture as GC and the same training parameters, see Sec.”
DVC: An End-To-End Deep Video Compression Framework (2018, methodology): “Other methods employed the CNNs to design an auto-encoder style network for image compression [11, 12, 33].”
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications (2022, positive): “DNN aided compression algorithms have achieved significant results, often outperforming state-of-the-art standardised codecs in a variety of source domains, from image [155], [156], [157], [158], [159], to video [160], [161], [162], [163], [164], [165], [166], [167], [168], speech [169], [170], and ”
Learned Image Compression With Discretized Gaussian Mixture Likelihoods and Attention Modules (2020, background): “In the early stage, some works are proposed to deal with non-differential quantization and rate estimation to make end-to-end training possible, such as [6, 7, 8].”
Video Compression through Image Interpolation (2018, background): “For images, deep networks yield state-of-the-art compression ratios with impressive reconstruction quality [6,12,22,25,26].”
Variable Rate Deep Image Compression With a Conditional Autoencoder (2019, background): “It was proposed in [3] to model the quantization error as additive uniform stochastic noise to relax the optimization of (2).”
End-to-End Neural Audio Coding in the MDCT Domain (2023, methodology): “The model is optimized via stochastic gradient descent to resolve the rate-distortion trade-off [8], where λ is a constant that determines the convergence point between the two criteria.”