Improving Inference for Neural Image Compression

By Yibo Yang, Robert Bamler, Stephan Mandt. 2020

144 citations recorded by the source corpus.

We consider the problem of lossy image compression with deep latent variable models. State-of-the-art methods build on hierarchical variational autoencoders (VAEs) and learn inference networks to predict a compressible latent representation of each data point. Drawing on the variational inference perspective on compression, we identify three approximation gaps which limit performance in the conventional approach: (i) an amortization gap, (ii) a discretization gap, and (iii) a marginalization gap. We propose improvements to each of these three shortcomings based on ideas related to iterative inference, stochastic annealing for discrete optimization, and bits-back coding, resulting in the first application of bits-back coding to lossy compression. In our experiments, which include extensive baseline comparisons and ablation studies, we achieve new state-of-the-art performance on lossy image compression using an established VAE architecture, by changing only the inference method.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Efficiency & Compression

Citations in the corpus

Evaluating Lossy Compression Rates of Deep Generative Models (2020, background): “Other researchers have developed practical lossy coding schemes achieving variational rate distortion bounds for particular latent variable models which exploited the factorial structure of the variational posterior (Ballé et al., 2018; Theis et al., 2017; Yang et al., 2020).”
Towards Backward-Compatible Continual Learning of Image Compression (2024, methodology): “Therefore, ex-25564 isting continual learning methods for vision tasks [13, 51] cannot be applied as is, and new strategies must be devel oped for compression to maintain the decoder's backward compatibility when adapting to new data and rates.”
Lossy Compression for Lossless Prediction (2021, methodology): “Further recent studies improve the compressor by improving the inference procedure of these models (Yang et al., 2020).”
ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding (2022, unknown): “To painlessly improve the coding efficiency, several training, inference, and encoding-time optimizing approaches are proposed [23, 52, 54].”