Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs

By Xiaohan Ding, X. Zhang, Yizhuang Zhou, Jungong Han, Guiguang Ding, Jian‐Yuan Sun. Computer Vision and Pattern Recognition, 2022

781 citations recorded by the source corpus.

We revisit large kernel design in modern convolutional neural networks (CNNs). Inspired by recent advances in vision transformers (ViTs), in this paper, we demonstrate that using a few large convolutional kernels instead of a stack of small kernels could be a more powerful paradigm. We suggested five guidelines, e.g., applying re-parameterized large depthwise convolutions, to design efficient high-performance large-kernel CNNs. Following the guidelines, we propose RepLKNet, a pure CNN architecture whose kernel size is as large as 31×31, in contrast to commonly used 3×3. RepLKNet greatly closes the performance gap between CNNs and ViTs, e.g., achieving comparable or superior results than Swin Transformer on ImageNet and a few typical downstream tasks, with lower latency. RepLKNet also shows nice scalability to big data and large models, obtaining 87.8% top-1 accuracy on ImageNet and 56.0% mIoU on ADE20K, which is very competitive among the state-of-the-arts with similar model sizes. Our study further reveals that, in contrast to small-kernel CNNs, large-kernel CNNs have much larger effective receptive fields and higher shape bias rather than texture bias. Code & models at https://github.com/megvii-research/RepLKNet.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Neurology

View Paper: https://www.semanticscholar.org/paper/9f1b0e4c42a5a85d4c023030557ade4419f82ecf

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Neurology

Citations in the corpus

Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks (2023, background): “There are a few other works [1, 11] also looking into the FLOPS issue and trying to improve it.”
Visual attention network (2022, methodology): “We notice some parallel works, which also adopt large kernel convolution such as ConvNeXt [21], and RepLKNet [39].”
TTST: A Top-k Token Selective Transformer for Remote Sensing Image Super-Resolution (2024, background): “Recently, Ding et al. [59] even introduced 31 × 31 convolutions and achieved competitive performance with a vision transformer.”
Large Selective Kernel Network for Remote Sensing Object Detection (2023, methodology): “In addition, RepLKNet [13] even uses a 31 × 31 convolutional kernel via re-parameterization, achieving compelling performance.”
HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions (2022, methodology): “RepLKNet [14] explores CNN models with very large kernels (up to 31×31), showing good scalability as vision Transformers.”
TransNeXt: Robust Foveal Visual Perception for Vision Transformers (2023, positive): “We believe that these experimental results are due to the introduction of pooling features provides coarse-grained global perception abilities, greatly reducing the demand for single queries to perceive the sliding window field.”
RTMDet: An Empirical Study of Designing Real-Time Object Detectors (2022, methodology): “Specifically, we first exploit large-kernel depth-wise convolutions in the basic building block of the backbone and neck in the model, which improves the model’s capability of capturing the global context [15].”
YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors (2022, methodology): “Model re-parametrization techniques [71, 31, 75, 19, 33, 11, 4, 24, 13, 12, 10, 29, 14, 78] merge multiple computational modules into one at inference stage.”
Large Separable Kernel Attention: Rethinking the Large Kernel Attention Design in CNN (2023, methodology): “However, as pointed out in [35], the naive design of large-scale depth-wise convolutional kernel still incurs high computational and memory footprints downgrading the effectiveness of the model with increasing kernel size.”
MetaFormer Baselines for Vision (2022, methodology): “The power of MetaFormer can also be verified by the recent models adopting MetaFormer as the general architecture but with different attention-based [40, 57, 70], MLP-based [66, 67, 27, 6, 41], convolution-based [15, 19, 79, 56, 77], hybrid [61, 12, 37, 70] or other types of [64, 20] token mixers.”