ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks

By Xiaohan Ding, Yuchen Guo, Guiguang Ding, Jungong Han. 2019

870 citations recorded by the source corpus.

As designing appropriate Convolutional Neural Network (CNN) architecture in the context of a given application usually involves heavy human works or numerous GPU hours, the research community is soliciting the architecture-neutral CNN structures, which can be easily plugged into multiple mature architectures to improve the performance on our real-world applications. We propose Asymmetric Convolution Block (ACB), an architecture-neutral structure as a CNN building block, which uses 1D asymmetric convolutions to strengthen the square convolution kernels. For an off-the-shelf architecture, we replace the standard square-kernel convolutional layers with ACBs to construct an Asymmetric Convolutional Network (ACNet), which can be trained to reach a higher level of accuracy. After training, we equivalently convert the ACNet into the same original architecture, thus requiring no extra computations anymore. We have observed that ACNet can improve the performance of various models on CIFAR and ImageNet by a clear margin. Through further experiments, we attribute the effectiveness of ACB to its capability of enhancing the model's robustness to rotational distortions and strengthening the central skeleton parts of square convolution kernels.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Initialization and Regularization of Factorized Neural Layers (2021, methodology): “, 2020) or using convolution-specific methods (Ding et al., 2019; Cao et al., 2020).”
Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning (2022, background): “Further improvements have been investigated for superior dynamic performance [19] and inference efficiency [5, 6, 9] recently.”
Representation Compensation Networks for Continual Semantic Segmentation (2022, methodology): “Inspired by structural re-parameterization [24, 25], we replace the convolution layers in the network with two parallel branches during training, which is named as representation compensation module.”
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers (2020, methodology): “The previous best method, ACNet [16] achieves mIoU 45.”
Diverse Branch Block: Building a Convolution as an Inception-like Unit (2021, methodology): “Following ACNet [8], we replace the two hidden fully-connected (FC) layers by global average”
ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks (2018, background): “[11] also advocates for expansion of convolutional layers.”
FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization (2023, methodology): “In order to increase capacity of the these layers, we use linear train-time overparameterization as introduced in [13, 11, 12, 57, 18].”
Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs (2022, methodology): “See [28,31] for details. ture [8,36].”
HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation (2022, neutral): “Several architectures [3,4,16,38,83] utilize intermediate features with different scales, which are generated from a single scale input, to aggregate context information.”
COVID-19 Detection Through Transfer Learning Using Multimodal Imaging Data (2020, background): “Recent studies attempted to improve performance of the base models by proposing new layers and ﬁlters such as Sparse Shift Filter [77], Asymmetric Convolution Block [78], Adder Networks [79], Virtual Pooling [80], Discrete Wavelet Transform [81], and HetConv [82], etc.”
YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors (2022, methodology): “Model re-parametrization techniques [71, 31, 75, 19, 33, 11, 4, 24, 13, 12, 10, 29, 14, 78] merge multiple computational modules into one at inference stage.”
Orthogonal Over-Parameterized Training (2020, methodology): “[11] overparameterizes a 2D convolution kernel by combining a 2D kernel of the same size and two additional 1D asymmetric kernels.”
Endpoints Weight Fusion for Class Incremental Semantic Segmentation (2023, methodology): “ACNet [14] and RepVGG [15] first propose structural re-parameterization, which is used to merge multi-branch Convolution-Batch Normalization serial sequences to a plain Convolution layer.”
RepVGG: Making VGG-style ConvNets Great Again (2021, background): “The comparison with ACB suggests the success of RepVGG should not be simply attributed to the effect of over-parameterization of every component, since ACB uses more parameters but yields inferior performance.”
DO-Conv: Depthwise Over-Parameterized Convolutional Layer (2020, methodology): “We found that while some H s exhibit strong skeleton pattern (major di ﬀ erences on center row and column) as observed in ACNet [10], other H s exhibit rather di ﬀ erent patterns, such as “Stage2 Conv5” and “Stage3 Conv5” in Figure 6 (B).”
UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition (2023, methodology): “After training, with the Structural Re-parameterization [13–18] methodology, we merge the BN layers into the conv layers so the small-kernel conv can be equivalently merged into the large-kernel one for inference.”
MobileOne: An Improved One millisecond Mobile Backbone (2022, methodology): “Like prior works on structural re-parameterization [11–13], MobileOne introduces linear branches at train-time which get re-parameterized at inference.”