ThunderNet: Towards Real-Time Generic Object Detection on Mobile Devices

By Zheng Qin, Zeming Li, Zhaoning Zhang, Yiping Bao, Gang Yu, Yuxing Peng, Jian‐Yuan Sun. 2019

286 citations recorded by the source corpus.

Real-time generic object detection on mobile platforms is a crucial but challenging computer vision task. Prior lightweight CNN-based detectors are inclined to use one-stage pipeline. In this paper, we investigate the effectiveness of two-stage detectors in real-time generic detection and propose a lightweight two-stage detector named ThunderNet. In the backbone part, we analyze the drawbacks in previous lightweight backbones and present a lightweight backbone designed for object detection. In the detection part, we exploit an extremely efficient RPN and detection head design. To generate more discriminative feature representation, we design two efficient architecture blocks, Context Enhancement Module and Spatial Attention Module. At last, we investigate the balance between the input resolution, the backbone, and the detection head. Benefit from the highly efficient backbone and detection part design, ThunderNet surpasses previous lightweight one-stage detectors with only 40% of the computational cost on PASCAL VOC and COCO benchmarks. Without bells and whistles, ThunderNet runs at 24.1 fps on an ARM-based device with 19.2 AP on COCO. To the best of our knowledge, this is the first real-time detector reported on ARM platforms. Code will be released for paper reproduction.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection (2019, background): “…proposed to improve its performance, including architecture redesign and reform [4, 9, 5, 28, 30], context and attention mechanism [2, 51, 38, 7, 44], multi-scale training and testing [54, 41], training strategy and loss function [40, 52, 61, 17], feature fusion and enhancement [25, 32], better…”
DetNAS: Backbone Search for Object Detection (2019, background): “The importance of backbones is also demonstrated in DetNet [11], Deformable ConvNets v2 [29], ThunderNet [21] in real-time object detection, and HRNet [24] in keypoint detection.”
SM-NAS: Structural-to-Modular Neural Architecture Search for Object Detection (2019, background): “[39] also suggested that input resolution should match the capability of the backbone, which is not measurable in practice.”
CenterMask: Real-Time Anchor-Free Instance Segmentation (2019, background): “Recently, attention methods [13, 34, 40, 28] have been widely applied to object detections because it helps to focus on important features, but also suppress unnecessary ones.”
AugFPN: Improving Multi-Scale Feature Learning for Object Detection (2019, methodology): “The information loss can be mitigated by combining the global context feature [29] extracted by global pooling.”
Rethinking Transformer-based Set Prediction for Object Detection (2020, methodology): “[23] uses a Spatial Attention Module to re-weight feature maps for making foreground features standing out.”
CSPNet: A New Backbone that can Enhance Learning Capability of CNN (2019, methodology): “Model Size GPU CPU mGPU AP50 SNet146-Thunder [25] 320 248 32 - 40.”
A Review on Deep Learning in UAV Remote Sensing (2021, methodology): “In specific tasks, such as object detection, it is possible to develop architectural enhancements for this approach, such as the Context Enhanced Module (CEM) and the Spatial Attention Module (SAM) [155].”
Object Detection in 20 Years: A Survey (2019, unknown): “In addition to some general designing principles, such as “fewer channels and more layers” [131], some other methods have been proposed in recent years [132], [133], [134], [135], [136].”
Distilling Object Detectors via Decoupled Features (2021, background): “Designing a detector coupled with lightweight backbone network [55, 41, 45, 32, 61, 64] is also a trend for faster detection speed.”
Task-Oriented Feature Distillation (2020, methodology): “Various techniques have been proposed to tackle this issue, including pruning [15, 41, 72, 67, 39], quantization [35, 46, 11, 50], lightweight model de-sign [23, 49, 27], and knowledge distillation (KD) [22, 53, 70].”
A Survey of Deep Learning Techniques for Weed Detection from Images (2021, background): “, 1996), and sNET (Qin et al., 2019) in their research.”
Towards Large-Scale Small Object Detection: Survey and Benchmarks (2022, background): “…Feature Fusion: Deep CNN architecture produces hierarchical feature maps at different spatial resolutions, in which low-level features describe ﬁner details along with more localization cues, while high-level features capture richer semantic information [13], [43], [76], [82], [83], [84].”
Self-Distillation: Towards Efficient and Compact Neural Networks (2021, methodology): “Typical approaches include pruning [12], [13], [14], [15], [16], quantization [17], [18], [19], [20], light-weight neural network design [21], [22], [23], [24], [25], low rank factorization [26], [27] and knowledge distillation [28], [29], [30], [31].”