Learning Convolutional Networks for Content-Weighted Image Compression

By Mu Li, Wangmeng Zuo, Shuhang Gu, Debin Zhao, David Dapeng Zhang. 2017

418 citations recorded by the source corpus.

Lossy image compression is generally formulated as a joint rate-distortion optimization problem to learn encoder, quantizer, and decoder. Due to the non-differentiable quantizer and discrete entropy estimation, it is very challenging to develop a convolutional network (CNN)-based image compression system. In this paper, motivated by that the local information content is spatially variant in an image, we suggest that: (i) the bit rate of the different parts of the image is adapted to local content, and (ii) the content-aware bit rate is allocated under the guidance of a content-weighted importance map. The sum of the importance map can thus serve as a continuous alternative of discrete entropy estimation to control compression rate. The binarizer is adopted to quantize the output of encoder and a proxy function is introduced for approximating binary operation in backward propagation to make it differentiable. The encoder, decoder, binarizer and importance map can be jointly optimized in an end-to-end manner. And a convolutional entropy encoder is further presented for lossless compression of importance map and binary codes. In low bit rate image compression, experiments show that our system significantly outperforms JPEG and JPEG 2000 by structural similarity (SSIM) index, and can produce the much better visual result with sharp edges, rich textures, and fewer artifacts.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

High-Fidelity Generative Image Compression (2020, methodology): “density of the auto-encoder latents for more efficient arithmetic coding, using hierarchical priors, auto-regression with various context shapes, or a combination thereof [6, 31, 28, 39, 32, 26, 33].”
Generative Adversarial Networks for Extreme Learned Image Compression (2018, background): “To investigate the effect of the GAN term in our total loss, we train a baseline model with an MSE loss only (with the same architecture as GC and the same training parameters, see Sec.”
Semantic Communications: Principles and Challenges (2021, background): “Based on the fact that local information content of an image is spatially variant, a content-aware bit rate allocation method was proposed in [60].”
Conditional Probability Models for Deep Image Compression (2018, methodology): “While [9] uses a separate network for this purpose, we consider a simplified setting.”
DVC: An End-To-End Deep Video Compression Framework (2018, methodology): “Rate-distortion optimization technique was adopted in [11, 12, 33, 21] for higher compression efficiency by introducing the number of bits in the optimization procedure.”
Learned Image Compression With Discretized Gaussian Mixture Likelihoods and Attention Modules (2020, methodology): “2) Minnen [MSE] (NIPS18) [20] Lee [MSE] (ICLR19) [21] Ballé [MSE] (ICLR18) [19] Ballé [MS-SSIM] (ICLR18) [19] Li (CVPR2018) [15] HEVC-intra (BPG) JPEG2000 (OpenJPEG) JPEG”
Checkerboard Context Model for Efficient Learned Image Compression (2021, methodology): “cluding using more complex entropy models [25, 28, 20, 10], enhancing reconstruction quality with post-processing networks [39], enabling variable rate compression using a single model [11, 13], using content aware compression [22], and incorporating multi-scale or attention-based neural architectur”
Video Compression With Rate-Distortion Autoencoders (2019, methodology): “We extend this work to video compression by: i) proposing a gated conditional autoregressive prior using 2D convolutions [37] with, optionally, a recurrent neural net for better entropy estimation over time, ii) encoding/decoding multiple frames by using 3D convolutions, iii) simplifying the model a”
Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff (2019, background): “Specifically, a lot of research efforts have been devoted to optimizing the rate-distortion function (2) using deep nets (Toderici et al., 2016; 2017; Agustsson et al., 2017; Ballé et al., 2017; Minnen et al., 2018; Li et al., 2018).”
Adaptive Weighting Multi-Field-Of-View CNN for Semantic Segmentation in Pathology (2019, methodology): “There are several methods that can adaptively weight the effective channels [20], pixels (location) [30], and scales [15][35] in a single network.”
Practical Full Resolution Learned Lossless Image Compression (2018, background): “Recent learned approaches include [20, 23, 26], where shallow autoregressive models over latents are learned.”
Deep Generative Models for Distribution-Preserving Lossy Compression (2018, background): “Compressive autoencoders [3, 5, 6, 8, 13, 10] and recurrent neural networks (RNNs) [1, 2, 7] have emerged as the most popular DNN architectures for compression.”
Joint Autoregressive and Hierarchical Priors for Learned Image Compression (2018, methodology): “Many learning-based methods take the form of an autoencoder, and separate models are trained to target different bit rates instead of training a single recurrent model [5]–[7], [9], [11], [12], [14], [34], [35].”
End-to-End Learnt Image Compression via Non-Local Attention Optimization and Improved Context Modeling (2019, methodology): “in [12], [18], an explicit mask encapsulated in the compressed”
Multi-level Wavelet-CNN for Image Restoration (2018, background): “Recently, convolutional neural networks (CNNs) have also been extensively studied and achieved state-of-the-art performance in several representative image restoration tasks, such as single image super-resolution (SISR) [16, 29, 32], image denoising [57], image deblurring [58], and compressed imagin”