Learned Image Compression with Mixed Transformer-CNN Architectures

By Jinming Liu, Heming Sun, Jiro Katto. Computer Vision and Pattern Recognition, 2023

467 citations recorded by the source corpus.

Learned image compression (LIC) methods have exhibited promising progress and superior rate-distortion performance compared with classical image compression standards. Most existing LIC methods are Convolutional Neural Networks-based (CNN-based) or Transformer-based, which have different advantages. Exploiting both advantages is a point worth exploring, which has two challenges: 1) how to effectively fuse the two methods? 2) how to achieve higher performance with a suitable complexity? In this paper, we propose an efficient parallel Transformer-CNN Mixture (TCM) block with a controllable complexity to incorporate the local modeling ability of CNN and the non-local modeling ability of transformers to improve the overall architecture of image compression models. Besides, inspired by the recent progress of entropy estimation models and attention modules, we propose a channel-wise entropy model with parameter-efficient swin-transformer-based attention (SWAtten) modules by using channel squeezing. Experimental results demonstrate our proposed method achieves state-of-the-art rate-distortion performances on three different resolution datasets (i.e., Kodak, Tecnick, CLIC Professional Validation) compared to existing LIC methods. The code is at https://github.com/jmliu206/LIC_TCM.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a23a4b04c487585792ecf45bce4ea43aa71bac2f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Zhihao Duan
  2. Justin Yang
  3. Zhan Ma
  4. F. Zhu
  5. Jiangpeng He
  6. Ming-Tse Lu

Citations in the corpus