RepVGG: Making VGG-style ConvNets Great Again

By Xiaohan Ding, X. Zhang, Ningning Ma, Jungong Han, Guiguang Ding, Jian‐Yuan Sun. 2021

2,415 citations recorded by the source corpus.

We present a simple but powerful architecture of convolutional neural network, which has a VGG-like inference-time body composed of nothing but a stack of 3 × 3 convolution and ReLU, while the training-time model has a multi-branch topology. Such decoupling of the training-time and inference-time architecture is realized by a structural re-parameterization technique so that the model is named RepVGG. On ImageNet, RepVGG reaches over 80% top-1 accuracy, which is the first time for a plain model, to the best of our knowledge. On NVIDIA 1080Ti GPU, RepVGG models run 83% faster than ResNet-50 or 101% faster than ResNet-101 with higher accuracy and show favorable accuracy-speed trade-off compared to the state-of-the-art models like EfficientNet and RegNet. The code and trained models are available at https://github.com/megvii-model/RepVGG.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning (2022, methodology): “After training, we use the structural re-parameterization [6] to integrate the sidebranch information into the main branch losslessly, ensuring that the number of network parameters does not increase at the end of each phase.”
Representation Compensation Networks for Continual Semantic Segmentation (2022, methodology): “Inspired by structural re-parameterization [24, 25], we replace the convolution layers in the network with two parallel branches during training, which is named as representation compensation module.”
InceptionNeXt: When Inception Meets ConvNeXt (2023, positive): “To employ larger kernels, RepLKNet [14] proposes to utilize structural re-parameterization techniques [13, 81] to scale up kernel size to 31 × 31 ; VAN [18] sequentially stacks large-kernel depth-wise convolution (DW-Conv) and depth-wise dilation convolution to obtain 21 × 21 receptive filed;…”
ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis (2024, methodology): “After training , each of these two branches is transformed into a single depth-wise convolution layer by fusing the BN parameters into the convolution kernels (Ding et al., 2021).”
A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations (2023, neutral): “including neural network pruning ([19], [20]), low-rank factor-izations of the weight matrices ([21], [22]), quantization ([23], [24]), knowledge distillation ([25], [26]), neural architecture search ([27], [28]), and other techniques ([29], [30]).”
FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization (2023, methodology): “For the past decade, convolutional neural networks have been the standard architecture for vision models [21, 44, 66, 50, 51, 26, 47, 25, 36, 13, 57].”
ViM: Out-Of-Distribution with Virtual-logit Matching (2022, methodology): “cent BiT [20], and to the latest ViT-B16 [8], RepVGG [7], DeiT [33] and Swin Transformer [26].”
YOLO-based Object Detection Models: A Review and its Applications (2024, positive): “Planned re-parameterized convolution YOLOv7 model is inspired by re-parameterized convolution [89].”
YOLOv10: Real-Time End-to-End Object Detection (2024, methodology): “Besides, model scaling strategies [62, 61] and re-parameterization [11, 29] techniques are also investigated.”
Rep ViT: Revisiting Mobile CNN From ViT Perspective (2023, methodology): “Additionally, compared to 3x3 convolutions, larger convolution kernels are typically not highly optimized by compilers and computing libraries [12].”
Real-Time Intermediate Flow Estimation for Video Frame Interpolation (2020, background): “Thanks related researchers for the exploration of eﬃcient models [48,36,14].”
Brain Tumor Detection Based on Deep Learning Approaches and Magnetic Resonance Imaging (2023, background): “The Rep structure takes its cue from RepVGG [71] and includes a tailored residual design that simplifies actual estimations to a convolution.”
SCSA: Exploring the Synergistic Effects Between Spatial and Channel Attention (2024, positive): “Specifically, we follow the parameter configurations in the original papers [19, 46, 8, 33], except for the batch size and learning rate.”
YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications (2022, methodology): “In this regard, we take RepBlock [3] as the building block of our small networks.”
The SpeakIn System for VoxCeleb Speaker Recognition Challange 2021 (2021, background): “• RepVGG [13] Recent researches proposed a new way to construct ConvNets.”
YOLO-World: Real-Time Open-Vocabulary Object Detection (2024, background): “Several works [11, 23, 33, 52, 55] propose various architectures or designs for YOLO, including path aggregation networks [29], cross-stage partial networks [51], and re-parameterization [7], which further improve both speed and accuracy.”
ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation (2023, background): “During inference, the different domain-represented ViDAs can be projected into the pre-trained model by re-parameterization [14], which ensures no extra parameter increase and maintain the model plasticity.”