Channel-Wise Autoregressive Entropy Models for Learned Image Compression

By David C. Minnen, Saurabh Singh. 2020

563 citations recorded by the source corpus.

In learning-based approaches to image compression, codecs are developed by optimizing a computational model to minimize a rate-distortion objective. Currently, the most effective learned image codecs take the form of an entropy-constrained autoencoder with an entropy model that uses both forward and backward adaptation. Forward adaptation makes use of side information and can be efficiently integrated into a deep neural network. In contrast, backward adaptation typically makes predictions based on the causal context of each symbol, which requires serial processing that prevents efficient GPU / TPU utilization. We introduce two enhancements, channel-conditioning and latent residual prediction, that lead to network architectures with better rate-distortion performance than existing context-adaptive models while minimizing serial processing. Empirically, we see an average rate savings of 6.7% on the Kodak image set and 11.4% on the Tecnick image set compared to a context-adaptive baseline model. At low bit rates, where the improvements are most effective, our model saves up to 18% over the baseline and outperforms hand-engineered codecs like BPG by up to 25%.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Feng Zhu
2. Jiangpeng He
3. Jiro Katto
4. Simon Feltman
5. Zhihao Duan
6. Fabien Racap'e
7. Justin Yang
8. Jinming Liu

Citations in the corpus

• Learned Image Compression with Mixed Transformer-CNN Architectures (2023, methodology): “This shows that our model has better modeling ability compared to the CNN-based model.”
• ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding (2022, unknown): “Another case is progressive decoding preview, investigated in prior literature [40].”
• CompressAI: a PyTorch library and evaluation platform for end-to-end compression research (2020, methodology): “Minimizing the bit-stream size requires to learn shared probability models between the encoder and decoder (priors) [1, 13, 2], as well as using relaxation methods [14, 15, 13, 16] to approximate the nondifferentiable quantization of the latent values.”
• Checkerboard Context Model for Efficient Learned Image Compression (2021, background): “A very recent work [27] proposes a channel-wise autoregressive entropy model that aims to minimize the elementlevel serial processing in the context model of [26] and achieves SOTA RD performance when combined with latent residual prediction and round-based training.”
• Towards Backward-Compatible Continual Learning of Image Compression (2024, background): “Another line of research lies in designing the entropy model pz, such as autoregressive [22, 40, 41, 44] and hierarchical models [3, 14, 15, 25].”
• Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications (2022, unknown): “In recent years, DNN aided compression algorithms have achieved significant results, often outperforming state-of-theart standardised codecs in a variety of source domains, from image [156]–[160], to video [161]–[169], speech [170], [171], and audio [172], [173] compression.”
• QARV: Quantization-Aware ResNet VAE for Lossy Image Compression (2023, methodology): “Thisconclusionholdsforalltypesofautoregressivemodels, includingthestandard,rasterscanorderARmodel[8](fora constantimageresolution),thechannel-wiseARmodel[10], andtheCheckerboardcontextmodel[9].”