Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks

By Nick Johnston, Damien Vincent, David C. Minnen, Michele Covell, Saurabh Singh, Troy T. Chinen, Sung Jin Hwang, Joel Shor, George Toderici. 2017

401 citations recorded by the source corpus.

We propose a method for lossy image compression based on recurrent, convolutional neural networks that outperforms BPG (4:2:0), WebP, JPEG2000, and JPEG as measured by MS-SSIM. We introduce three improvements over previous research that lead to this state-of-the-art result using a single model. First, we modify the recurrent architecture to improve spatial diffusion, which allows the network to more effectively capture and propagate image information through the network's hidden state. Second, in addition to lossless entropy coding, we use a spatially adaptive bit allocation algorithm to more efficiently use the limited number of bits to encode visually complex image regions. Finally, we show that training with a pixel-wise loss weighted by SSIM increases reconstruction quality according to multiple metrics. We evaluate our method on the Kodak and Tecnick image sets and compare against standard codecs as well as recently published methods based on deep neural networks.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations (2017, methodology): “Concurrent to our work, the method of [16] builds on the architecture proposed in [33], and shows that impressive performance in terms of the MS-SSIM metric can be obtained by incorporating it into the optimization (instead of just minimizing the MSE).”
Cascaded Cross-Module Residual Learning towards Lightweight End-to-End Speech Coding (2019, methodology): “Motivated by DNN based end-to-end codecs [14] and residual cascading [22][23], this paper proposes a “cross-module” residual learning (CMRL) pipeline, which can lower the model complexity while maintaining a high perceptual quality and compression ratio.”
Learned Video Compression (2018, background): “In the last two years, we have seen a great surge of ML-based image compression approaches [13, 39, 40, 5, 4, 12, 20, 38, 33, 18, 2, 22, 6, 3, 8, 27, 28].”
Real-Time Adaptive Image Compression (2017, methodology): “2017) 3 in all settings in which results are available.”
Video Compression With Rate-Distortion Autoencoders (2019, background): “They often embed an input image into a low dimensional representation using fully convolutional [28] or recurrent networks [4, 22, 36].”
Variable Rate Deep Image Compression With a Conditional Autoencoder (2019, methodology): “Our model shows superior performance over the RNNbased variable-rate model in [9].”
End-to-End Learnt Image Compression via Non-Local Attention Optimization and Improved Context Modeling (2019, background): “An explicit spatially adaptive bit rate allocation was suggested in [34] to consider the content variations for better quality at the same bit rate.”
Soft-to-Hard Vector Quantization for End-to-End Learned Compression of Images and Neural Networks (2017, methodology): “Concurrent to our work, the method of (Johnston et al., 2017) builds on the architecture proposed in (Toderici et al., 2016), and shows that impressive performance on the perceptual MS-SSIM metric (Wang et al., 2003) can be obtained by incorporating it into the optimization (and using sophisticated…”
FVC: A New Framework towards Deep Video Compression in Feature Space (2021, methodology): “The RNN-based image compression methods [31, 32, 15] are firstly introduced to progressively compress images.”
Deep Joint Source-channel Coding for Wireless Image Transmission (2018, positive): “The next step in improving the performance of the deep JSCC scheme is to exploit more advanced NN architectures in the autoencoder that have been shown to improve the compression performance [6], [40].”
Online Learned Continual Compression with Stacked Quantization Module (2019, methodology): “Thus, learned compression has become a topic of great interest (Theis et al., 2017; Ballé et al., 2016; Johnston et al., 2018).”
Online Learned Continual Compression with Adaptive Quantization Modules (2019, methodology): “Works by Theis et al. (2017); Ball´e et al. (2016); Johnston et al. (2018) have shown learned compressions can outperform standard algorithms like JPEG.”
An Information-Theoretic Analysis of Deep Latent-Variable Models (2017, methodology): “Johnston et al. (2017) uses a recurrent VAE architecture to achieve state-of-the-art image compression rates, posing the loss as minimizing distortion at a ﬁxed rate.”
Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff (2019, neutral): “…application in this domain (Toderici et al., 2016; 2017; Ball´e et al., 2016; 2017; 2018; Agustsson et al., 2017; 2018; Rippel & Bour-dev, 2017; Minnen et al., 2018; Li et al., 2018; Mentzer et al., 2018; Johnston et al., 2018; Galteri et al., 2017; Tschannen et al., 2018; Santurkar et al., 2018).”
Joint Autoregressive and Hierarchical Priors for Learned Image Compression (2018, background): “Their approach was improved by exploring other recurrent architectures for the autoencoder, training an LSTM-based entropy model, and adding a post-process that spatially adapts the bit rate based on the complexity of the local image content [4], [8].”
Lossy Compression for Lossless Prediction (2021, methodology): “Later approaches improved upon the original model by adding a hierarchical layers and autoregressive components (Ballé et al., 2018; Minnen et al., 2018; Lee et al., 2019; Johnston et al., 2018).”