Rep ViT: Revisiting Mobile CNN From ViT Perspective

By Ao Wang, Hui Chen, Zijia Lin, Hengjun Pu, Guiguang Ding. Computer Vision and Pattern Recognition, 2023

639 citations recorded by the source corpus.

Recently, lightweight Vision Transformers (ViTs) demon-strate superior performance and lower latency, compared with lightweight Convolutional Neural Networks (CNNs), on resource-constrained mobile devices. Researchers have discovered many structural connections be-tween lightweight ViTs and lightweight CNNs. However, the notable architectural disparities in the block structure, macro, and micro designs between them have not been adequately examined. In this study, we revisit the efficient design of lightweight CNNs from ViT perspective and emphasize their promising prospect for mobile devices. Specifically, we incrementally enhance the mobile-friendliness of a standard lightweight CNN, i.e., MobileNetV3, by integrating the efficient architectural designs of lightweight ViTs. This ends up with a new family of pure lightweight CNNs, namely RepViT. Extensive experiments show that RepViT outperforms existing state-of-the-art lightweight ViTs and exhibits favorable latency in various vision tasks. Notably, on ImageNet, RepViT achieves over 80% top-1 accuracy with 1.0 ms latency on an iPhone 12, which is the first time for a lightweight model, to the best of our knowledge. Besides, when RepViT meets SAM, our RepViT-SAM can achieve nearly 10x faster inference than the advanced MobileSAM. Codes and models are available at https://github.com/THU-MIG/RepViT.

Subfields: Computer Networks and Communications, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/198fb07ae84ca36f98a362058fb8a90e214bea06

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

1. Jungong Han

In Computer Vision and Pattern Recognition

1. Zijia Lin
2. Guiguang Ding
3. Jungong Han
4. Hui Chen
5. Ao Wang
6. Kai Chen
7. Lihao Liu
8. Dacheng Tao

Citations in the corpus

• SCSA: Exploring the Synergistic Effects Between Spatial and Channel Attention (2024, neutral): “Currently, many studies develop efficient models [66, 35, 4, 53] based on multi-semantic ideas, reducing parameters and computation for enhanced inference efficiency.”
• YOLOv10: Real-Time End-to-End Object Detection (2024, methodology): “Additionally, we employ the structural reparameterization technique [11, 10, 59] to bring another 3 × 3 depthwise convolution branch to alleviate the optimization issue without inference overhead.”