Involution: Inverting the Inherence of Convolution for Visual Recognition
By Duo Li, Jie Hu, Changhu Wang, Xiangtai Li, Qi She, Lei Zhu, T. Zhang, Qifeng Chen. 2021
392 citations recorded by the source corpus.
Convolution has been the core ingredient of modern neural networks, triggering the surge of deep learning in vision. In this work, we rethink the inherent principles of standard convolution for vision tasks, specifically spatial-agnostic and channel-specific. Instead, we present a novel atomic operation for deep neural networks by inverting the aforementioned design principles of convolution, coined as involution. We additionally demystify the recent popular self-attention operator and subsume it into our involution family as an over-complicated instantiation. The proposed involution operator could be leveraged as fundamental bricks to build the new generation of neural networks for visual recognition, powering different deep learning models on several prevalent benchmarks, including ImageNet classification, COCO detection and segmentation, together with Cityscapes segmentation. Our involution-based models improve the performance of convolutional baselines using ResNet-50 by up to 1.6% top-1 accuracy, 2.5% and 2.4% bounding box AP, and 4.7% mean IoU absolutely while compressing the computational cost to 66%, 65%, 72%, and 57% on the above benchmarks, respectively. Code and pre-trained models for all the tasks are available at https://github.com/d-li14/involution.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Neil Houlsby
- Jianfeng Gao
- Xiaohua Zhai
- Quoc V. Le
- Lucas Beyer
- Alexander Kolesnikov
- Mohit Bansal
- Jianwei Yang
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Trevor Darrell
- Jitendra Malik
- Xiaogang Wang
- Ming-Hsuan Yang
- Andrew Zisserman
- Cordelia Schmid
In Model Architectures
- Jianfeng Gao
- Jianwei Yang
- Xiaohua Zhai
- Neil Houlsby
- Alexander Kolesnikov
- Lucas Beyer
- Thomas Unterthiner
- Alexey Dosovitskiy
Citations in the corpus
- MLP-Mixer: An all-MLP Architecture for Vision (2021, background): “[26], and Bello [3] design networks with new attention-like mechanisms.”
- TransNeXt: Robust Foveal Visual Perception for Vision Transformers (2023, neutral): “Linear complexity mode for inference : We observe that in Equations 15 and 16, if we consistently set H p and W p as fixed values independent of the input size, the computational complexity of both pixel-focused attention and aggregated attention grows linearly with the length of the input sequence.”
- Focal Modulation Networks (2022, methodology): “Introducing dynamic operation is another way to augment ConvNets as demonstrated in Involution [37] and DyConv [10].”