Improving Convolutional Networks With Self-Calibrated Convolutions

By Jiangjiang Liu, Qibin Hou, Ming-Ming Cheng, Changhu Wang, Jiashi Feng. Computer Vision and Pattern Recognition, 2020

421 citations recorded by the source corpus.

Recent advances on CNNs are mostly devoted to designing more complex architectures to enhance their representation learning capacity. In this paper, we consider how to improve the basic convolutional feature transformation process of CNNs without tuning the model architectures. To this end, we present a novel self-calibrated convolutions that explicitly expand fields-of-view of each convolutional layers through internal communications and hence enrich the output features. In particular, unlike the standard convolutions that fuse spatial and channel-wise information using small kernels (e.g., 3x3), self-calibrated convolutions adaptively build long-range spatial and inter-channel dependencies around each spatial location through a novel self-calibration operation. Thus, it can help CNNs generate more discriminative representations by explicitly incorporating richer information. Our self-calibrated convolution design is simple and generic, and can be easily applied to augment standard convolutional layers without introducing extra parameters and complexity. Extensive experiments demonstrate that when applying self-calibrated convolutions into different backbones, our networks can significantly improve the baseline models in a variety of vision tasks, including image recognition, object detection, instance segmentation, and keypoint detection, with no need to change the network architectures. We hope this work could provide a promising way for future research in designing novel convolutional feature transformations for improving convolutional networks. Code is available on the project page.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d920e0710d2a1ae966661d0513b817b6b81dc2b2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Large Selective Kernel Network for Remote Sensing Object Detection (2023, background): “Similarly to the SKNet, SCNet [37] uses branch attention to capturing richer information and spatial attention to improve localization ability.”
LayerCAM: Exploring Hierarchical Class Activation Maps for Localization (2021, methodology): “Researchers have proposed many attention methods [11], [12], [13], [14], [15] to locate the object regions of the class of interest from the powerful CNN based image classiﬁers [16], [17], [18], [19], [20], [21], [22].”
SCSA: Exploring the Synergistic Effects Between Spatial and Channel Attention (2024, neutral): “Existing plug-and-play attention methods can be primarily categorized into three types: channel attention [23, 27, 62, 55, 31, 41], spatial attention [22, 56, 42, 9], and hybrid channel-spatial attention [57, 40, 38, 14, 20, 65, 39, 61].”
Review of Image Classification Algorithms Based on Convolutional Neural Networks (2021, background): “In addition, GSoP-Net [142], A2Net [148], NLNet [154], GCNet [155], SCNet [156] and CCNet [157] serve as typical examples of non-local/self-attention networks are recently very popular due to their capability of building spatial or channel-wise attention.”
Efficient Image Super-Resolution Using Pixel Attention (2020, methodology): “tacked Self-Calibrated convolution with Pixel Attention (SC-PA) blocks. Here, we dene x n 1 and x n as the input and output of the nth SC-PA block, respectively. As shown in Fig.2, similarly as SCNet[21], SC-PA block contains two branches, where each branch has a 1 1 convolution layer at the beginni”
Attention mechanisms in computer vision: A survey (2021, methodology): “Motivated by the success of group convolution, Liu et at [86] presented self-calibrated convolution as a means to enlarge the receptive field at each spatial location.”
DeepViT: Towards Deeper Vision Transformer (2021, background): “The recent progress of deep CNN models is largely driven by training very deep models with a large number of layers which is enabled by novel model architecture designs [9, 41, 31, 20, 47].”
Rotate to Attend: Convolutional Triplet Attention Module (2020, background): “rs of computer vision research, convolutional neural network architectures of increasing depth have demonstrated major success in many computer vision tasks [13, 18, 29, 30, 37]. Numerous recent work [5, 15, 25, 34, 24] have proposed using either channel attention, or spatial attention, or both to i”
Coordinate Attention for Efficient Mobile Network Design (2021, background): “Typical examples include NLNet [43], GC-Net [3], A 2 Net [7], SCNet [25], GSoP-Net [11], or CC-Net [19], all of which exploit non-local mechanisms to capture different types of spatial information.”
AIM 2020 Challenge on Real Image Super-Resolution: Methods and Results (2020, background): “to the learning of the high frequency parts in images. As shown in Fig.4, three Basic Residual Block (BRB) forms a Large Residual Block (LRB) with dense connection. Self-Calibration convolution (SCC) [20], shown at top of Fig.4, is adopted as a basic unit in order to expand receptive eld. Unlike con”
Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition (2021, background): “Attention mechanisms (Hu et al., 2018; 2019; Wang et al., 2018; Bello et al., 2019; Liu et al., 2020; Chen et al., 2018) break through the limitations of convolutions in capturing local features and further promote the development of CNNs.”