The Devil Is in the Details: Window-based Attention for Image Compression

By Renjie Zou, Chunfeng Song, Zhaoxiang Zhang. Computer Vision and Pattern Recognition, 2022

347 citations recorded by the source corpus.

Learned image compression methods have exhibited superior rate-distortion performance than classical image compression standards. Most existing learned image compression models are based on Convolutional Neural Networks (CNNs). Despite great contributions, a main drawback of CNN based model is that its structure is not designed for capturing local redundancy, especially the nonrepetitive textures, which severely affects the reconstruction quality. Therefore, how to make full use of both global structure and local texture becomes the core problem for learning-based image compression. Inspired by recent progresses of Vision Transformer (ViT) and Swin Transformer, we found that combining the local-aware attention mechanism with the global-related feature learning could meet the expectation in image compression. In this paper, we first extensively study the effects of multiple kinds of attention mechanisms for local features learning, then introduce a more straightforward yet effective window-based local attention block. The proposed window-based attention is very flexible which could work as a plug-and-play component to enhance CNN and Transformer models. Moreover, we propose a novel Symmetrical TransFormer (STF) framework with absolute transformer blocks in the down-sampling encoder and up-sampling decoder. Extensive experimental evaluations have shown that the proposed method is effective and outperforms the state-of-the-art methods. The code is publicly available at https://github.com/Googolxx/STF.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/415df1f3d442b9a20cfd6e07bb57550624fffc0b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Jiangpeng He
2. Feng Zhu
3. Jiro Katto
4. Zhihao Duan
5. Justin Yang
6. Jinming Liu
7. Jackie Ma
8. Zhan Ma

Citations in the corpus

• Learned Image Compression with Mixed Transformer-CNN Architectures (2023, methodology): “We conducted a comparison between our TCM-based model and both a CNN-based [6] and Transformer-based [37] model using the Kodak dataset’s kodim 19 and kodim 20 images.”
• Towards Backward-Compatible Continual Learning of Image Compression (2024, background): “A major group focuses on designing expressive architectures for !enc and f ctec, such as convolutional and transformer-based ones [10, 11, 14, 20, 23, 27, 33, 37-39, 43, 52, 54, 59].”
• QARV: Quantization-Aware ResNet VAE for Lossy Image Compression (2023, background): “…efficiency, existing works have explored efficient neural network blocks, such as residual networks [21], self-attention [22], and transformers [23], [24], [25], as well as developed more expressive entropy models, such as hierarchical [7], [26] and autoregressive models [8], [9], [10], [27].”