ESPNetv2: A Light-Weight, Power Efficient, and General Purpose Convolutional Neural Network

By Sachin Mehta, Mohammad Rastegari, Linda G. Shapiro, Hannaneh Hajishirzi. Computer Vision and Pattern Recognition, 2018

473 citations recorded by the source corpus.

We introduce a light-weight, power efficient, and general purpose convolutional neural network, ESPNetv2, for modeling visual and sequential data. Our network uses group point-wise and depth-wise dilated separable convolutions to learn representations from a large effective receptive field with fewer FLOPs and parameters. The performance of our network is evaluated on four different tasks: (1) object classification, (2) semantic segmentation, (3) object detection, and (4) language modeling. Experiments on these tasks, including image classification on the ImageNet and language modeling on the PenTree bank dataset, demonstrate the superior performance of our method over the state-of-the-art methods. Our network outperforms ESPNet by 4-5% and has 2-4x fewer FLOPs on the PASCAL VOC and the Cityscapes dataset. Compared to YOLOv2 on the MS-COCO object detection, ESPNetv2 delivers 4.4% higher accuracy with 6x fewer FLOPs. Our experiments show that ESPNetv2 is much more power efficient than existing state-of-the-art efficient methods including ShuffleNets and MobileNets. Our code is open-source and available at https://github.com/sacmehta/ESPNetv2.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/c253c9a7822d362405f4bc6bafe3b9746c21c142

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Quoc V. Le
2. Mingxing Tan
3. Chunhua Shen
4. Mohammad Rastegari
5. Alan L. Yuille
6. Song Han
7. Jiashi Feng
8. Liang-Chieh Chen

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Chunhua Shen
3. Dacheng Tao
4. Ming-Hsuan Yang
5. Xiaogang Wang
6. Alan L. Yuille
7. Yi Yang
8. Jiashi Feng

Citations in the corpus

• BiSeNet V2: Bilateral Network with Guided Aggregation for Real-Time Semantic Segmentation (2020, methodology): “Meanwhile, ESP-Net (Mehta et al., 2018, 2019) devises an efficient spatial pyramid dilated convolution for real-time semantic segmentation.”
• A systematic review of Green AI (2023, methodology): “tic edge inference (Kim & Wu, 2020), and implementations of convolutional neural networks optimized for energy efficiency (Mehta et al., 2019).”
• Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis (2020, methodology): “Compared to ERFNet, LEDNet, and ESPNetv2, SwiftNet [30] is both faster and achieves higher mIoU.”
• DKM: Differentiable K-Means Clustering Layer for Neural Network Compression (2021, methodology): “Efficient networks: Efficient architectures such MobileNet [8, 16], Efficient Net [21, 22] and ESPNet [11] are designed to be memory-efficient.”
• A Noise-Robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions From CT Images (2020, background): “…the network to focus more on the target region; 3) ScSE U-Net [46] that combines concurrent Spatial and Channel SE (ScSE) blocks with U-Net [16]; 4) ESPNetv2 [47] that is a light-weight, power efficient and general purpose CNN and achieved state-of-the-art performance for semantic segmentation.”
• Pixel Difference Networks for Efficient Edge Detection (2021, background): “Alternatively, lightweight architectures for other dense prediction tasks [13, 58, 43, 25, 38, 63] and multi-task learning [24, 26] may also benefit edge detection.”
• Deep Learning for Unmanned Aerial Vehicle-Based Object Detection and Tracking: A survey (2021, background): “Similarly, there are some faster lightweight networks, such as Mobilenet SSD [27], YOLOv3 [28], Efficient Spatial Pyramid [29], and so on.”
• Searching for MobileNetV3 (2019, methodology): “Our segmentation models with MobileNetV3 as network backbone outperforms ESPNetv2 [32], CCC2 [34], and ESPNetv1 [32] by 6.4%, 10.6%, 12.3%, respectively while being faster in terms of MAdds.”
• EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications (2022, methodology): “ESPNetv2 [30] utilizes depth-wise dilated convolutions to increase the receptive field of the network without increasing the network complexity.”
• Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks (2019, methodology): “In a real environment, several strategies can be considered to avoid this, such as replacing the backbone with MobileNet V2 [107], dimension reduction [108], or using the recently released ESPNet V2 [109] models.”
• MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer (2021, methodology): “Following a standard training practice (e.g., Chen et al., 2017; Mehta et al., 2019), we also use extra annotations and data from Hariharan et al. (2011) and Lin et al. (2014), respectively.”
• Semantic Flow for Fast and Accurate Scene Parsing (2020, methodology): “ESP-Nets [32, 33] save computation by decomposing standard convolution into point-wise convolution and spatial pyramid of dilated convolutions.”
• TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation (2022, methodology): “ESPNets [32] save computation by decomposing standard convolution into pointwise convolution and spatial pyramid of dilated convolutions.”
• MiniVLM: A Smaller and Faster Vision-Language Model (2020, unknown): “[37, 48, 52, 33, 20].”