Conditional Probability Models for Deep Image Compression

By Fabian Mentzer, Eirikur Agustsson, Michael Tschannen, Radu Timofte, Luc Van Gool. 2018

540 citations recorded by the source corpus.

Deep Neural Networks trained as image auto-encoders have recently emerged as a promising direction for advancing the state-of-the-art in image compression. The key challenge in learning such networks is twofold: To deal with quantization, and to control the trade-off between reconstruction error (distortion) and entropy (rate) of the latent image representation. In this paper, we focus on the latter challenge and propose a new technique to navigate the rate-distortion trade-off for an image compression auto-encoder. The main idea is to directly model the entropy of the latent representation by using a context model: A 3D-CNN which learns a conditional probability model of the latent distribution of the auto-encoder. During training, the auto-encoder makes use of the context model to estimate the entropy of its representation, and the context model is concurrently updated to learn the dependencies between the symbols in the latent representation. Our experiments show that this approach, when measured in MS-SSIM, yields a state-of-the-art image compression system based on a simple convolutional auto-encoder.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

High-Fidelity Generative Image Compression (2020, methodology): “At the same time, deep learning-based lossy compression has seen great interest [45, 5, 31], where a neural network is directly optimized for the rate-distortion trade-off, which led to new state-of-the-art methods.”
Learned Video Compression (2018, background): “In the last two years, we have seen a great surge of ML-based image compression approaches [13, 39, 40, 5, 4, 12, 20, 38, 33, 18, 2, 22, 6, 3, 8, 27, 28].”
Generative Adversarial Networks for Extreme Learned Image Compression (2018, methodology): “The so-obtained model has a bitrate of 0.07 bpp and gets a slightly better MS-SSIM than BPG at the same bpp on the validation set.”
DVC: An End-To-End Deep Video Compression Framework (2018, methodology): “In addition, other techniques such as generalized divisive normalization (GDN) [11], multi-scale image decomposition [28], adversarial training [28], importance map [21, 24] and intra prediction [25, 10] are proposed to improve the image compression performance.”
Variable Rate Deep Image Compression With a Conditional Autoencoder (2019, methodology): “We compare the performance of our variable-rate model to the state-of-the-art learned image compression models from [4, 9, 14–16, 19] and the classical state-of-the-art variable-rate image compression codec, BPG [5], on the Kodak image set [12].”
Finite Scalar Quantization: VQ-VAE Made Simple (2023, methodology): “Bounded scalar quantization ( i.e. , FSQ), has been used to represent images with high ﬁdelity (Mentzer et al. (2018) use d =16 , L =5 ), and for “extreme compression” (Agustsson et al. (2019) used d =5 , L =5 ).”
Video Compression through Image Interpolation (2018, methodology): “We follow Mentzer et al. [17] and use adaptive arithmetic coding on probability estimates of a Pixel-CNN [20].”
The Devil Is in the Details: Window-based Attention for Image Compression (2022, unknown): “Recently, learned image compression models [5,6,15–17, 19, 27 , 32, 34, 35 ] based on CNNs exhibit a fast development trend Transformer-based Model.”
Deep Generative Models for Distribution-Preserving Lossy Compression (2018, methodology): “Here, we simply fix them to the set of vectors {−1, 1}R and use the differentiable approximation from [9] to backpropagate gradients through this non-differentiable embedding.”
End-to-End Learnt Image Compression via Non-Local Attention Optimization and Improved Context Modeling (2019, methodology): “ing [12], [18] to do adaptive bit allocation, existing approaches are too simple to handle images with complex content.”
Unified Multivariate Gaussian Mixture for Efficient Neural Image Compression (2022, methodology): “Specifically, latents are quantized by rounding to the nearest integer [3] or prototype [25] in order to perform entropy coding with e.”
Practical Full Resolution Learned Lossless Image Compression (2018, methodology): “We use the scalar quantization approach proposed in [23] to quantize the output of E ( s ) : Given levels L = { 1 , . . . to , (cid:96) L } ⊂ R , we use nearest neighbor assignments quantize each entry z ∈ z ( s ) as but use differentiable “soft quantization” to compute gradients for the backward…”
Video Compression With Rate-Distortion Autoencoders (2019, methodology): “The closest to us is the rate-distortion autoencoder proposed in [28] for image compression.”
Learned Image Compression With Discretized Gaussian Mixture Likelihoods and Attention Modules (2020, background): “Recently, several studies investigate the adaptive context model for entropy estimation to navigate the optimization process of neural network parameters to achieve best tradeoff between reconstruction errors and required bits (entropy), including [19, 20, 21, 22].”
Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff (2019, methodology): “To enable back-propagation through the quantizer, we use the differentiable relaxation of (Mentzer et al., 2018).”
Scale-Space Flow for End-to-End Optimized Video Compression (2020, methodology): “Image Compression Research on learning-based image compression [7, 10, 32, 4, 29, 8, 25, 5] has shown significant progress in terms of rate–distortion performance compared to standard codecs such as JPEG [34], JPEG2000 [21] and BPG [11].”
Channel-Wise Autoregressive Entropy Models for Learned Image Compression (2020, background): “Backward-adaptation, on the other hand, typically incorporates predictions from the causal context of each symbol, i.e . neighboring symbols above and to the left of the current symbol as well as symbols in previously decoded channels [11]–[14].”