MAXIM: Multi-Axis MLP for Image Processing

By Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, A. Bovik, Yinxiao Li. Computer Vision and Pattern Recognition, 2022

716 citations recorded by the source corpus.

Recent progress on Transformers and multilayer perceptron (MLP) models provide new network architectural designs for computer vision tasks. Although these models proved to be effective in many vision tasks such as image recognition, there remain challenges in adapting them for lowlevel vision. The inflexibility to support high-resolution images and limitations of local attention are perhaps the main bottlenecks. In this work, we present a multi-axis MLP based architecture called MAXIM, that can serve as an efficient and flexible general-purpose vision backbone for image processing tasks. MAXIM uses a UNet-shaped hierarchical structure and supports long-range interactions enabled by spatially-gated MLPs. Specifically, MAXIM contains two MLP-based building blocks: a multi-axis gated MLP that allows for efficient and scalable spatial mixing of local and global visual cues, and a cross-gating block, an alternative to cross-attention, which accounts for cross-feature conditioning. Both these modules are exclusively based on MLPs, but also benefit from being both global and ‘fully-convolutional’, two properties that are desirable for image processing. Our extensive experimental results show that the proposed MAXIM model achieves state-of-the-art performance on more than ten benchmarks across a range of image processing tasks, including denoising, deblurring, de raining, dehazing, and enhancement while requiring fewer or comparable numbers of parameters and FLOPs than competitive models. The source code and trained models will be available at https://github.com/google-research/maxim.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/fbf68eb0cf8237cef59ebdb301569c79b9676ff9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Simple Baselines for Image Restoration (2022, methodology): “Image Deblurring with JPEG artifacts We conduct experiments on REDS[26] dataset, the training setting follows [4,30], and we evaluate the result on 300 images from the validation set (noted as REDS-val-300) following [4,30].”
Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring (2022, methodology): “The results are obtained from the Test100 dataset [16] Methods MPRNet [15] KiT [5] Restormer [14] MAXIM [”
MaxViT: Multi-Axis Vision Transformer (2022, background): “modules following [54,84], while there also exist other models that adopt a parallel design [81,98].”
V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer (2022, methodology): “To ameliorate this issue, numerous methods have introduced locality into self-attention, such as Swin [25], CSwin [7], Twins [6], window [40,34], and sparse attention [37,35,43].”
Activating More Pixels in Image Super-Resolution Transformer (2022, background): “Due to the impressive performance, Transformer has also been introduced for low-level vision tasks [4, 6, 27, 30, 31, 51, 57, 65].”
Dual Aggregation Transformer for Image Super-Resolution (2023, methodology): “Moreover, different from previous works [22, 5, 36], our SG module utilizes depthwise convolution to maintain computational efficiency.”
CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers (2022, background): “Recent works have been focused on improving the architectures of attention models, including sparse attention [33, 34, 35, 36, 37, 38, 39], enlarged receptive fields [40, 41], pyramidal designs [42, 43, 44], efficient alternatives [45, 46, 47], etc.”
Image Restoration with Mean-Reverting Stochastic Differential Equations (2023, methodology): “For comparison, we report four milestone deblur-ring approaches: DeepDeblur (Nah et al., 2017), Deblur-GAN (Kupyn et al., 2018), DeblurGAN-v2 (Kupyn et al., 2019), DBGAN (Zhang et al., 2020), and MAXIM (Tu et al., 2022).”
Restoring Vision in Adverse Weather Conditions With Patch-Based Denoising Diffusion Models (2022, background): “3238179 [7] and multi-layer perceptron based models [8].”