EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications

By Muhammad Maaz, Abdelrahman M. Shaker, Hisham Cholakkal, Salman Hameed Khan, Syed Waqas Zamir, Rao Muhammad Anwer, Fahad Shahbaz Khan. ECCV Workshops, 2022

374 citations recorded by the source corpus.

In the pursuit of achieving ever-increasing accuracy, large and complex neural networks are usually developed. Such models demand high computational resources and therefore cannot be deployed on edge devices. It is of great interest to build resource-efficient general purpose networks due to their usefulness in several application areas. In this work, we strive to effectively combine the strengths of both CNN and Transformer models and propose a new efficient hybrid architecture EdgeNeXt. Specifically in EdgeNeXt, we introduce split depth-wise transpose attention (STDA) encoder that splits input tensors into multiple channel groups and utilizes depth-wise convolution along with self-attention across channel dimensions to implicitly increase the receptive field and encode multi-scale features. Our extensive experiments on classification, detection and segmentation tasks, reveal the merits of the proposed approach, outperforming state-of-the-art methods with comparatively lower compute requirements. Our EdgeNeXt model with 1.3M parameters achieves 71.2% top-1 accuracy on ImageNet-1K, outperforming MobileViT with an absolute gain of 2.2% with 28% reduction in FLOPs. Further, our EdgeNeXt model with 5.6M parameters achieves 79.4% top-1 accuracy on ImageNet-1K. The code and models are available at https://t.ly/_Vu9.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/05b7bd47fa5cbe10497c49004b57eb5ab4fdd0b4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Transformer-Based Visual Segmentation: A Survey (2023, background): “Several earlier works [322], [323], [324], [325], [326], [327] focus on a more efﬁcient transformer backbone.”
Rethinking Vision Transformers for MobileNet Size and Speed (2022, background): “Another track is to combine lightweight CNN and attention mechanism to form a hybrid architecture [56, 13, 55].”
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review (2023, methodology): “A hybrid DL model called EdgeNeXt is presented in [71], which combines the power of CNN with the self-attention mechanism of a transformer for computer vision tasks considering the resource constraints of edge devices.”
Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation (2022, methodology): “With the rise of vision transformer (ViT) [8] recent work applied it to various computer vision tasks [4, 16, 29, 31, 34], and achieved promising results.”
UNETR++: Delving Into Efficient and Accurate 3D Medical Image Segmentation (2022, background): “Few recent works have explored alleviating the complexity issue of standard self-attention operation within transformer frameworks [7, 18, 23, 32].”