M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Network

By Qijie Zhao, Tao Sheng, Yongtao Wang, Zhi Tang, Ying Chen, Lingyi Cai, Haibin Ling. 2018

868 citations recorded by the source corpus.

Feature pyramids are widely exploited by both the state-of-the-art one-stage object detectors (e.g., DSSD, RetinaNet, RefineDet) and the two-stage object detectors (e.g., Mask RCNN, DetNet) to alleviate the problem arising from scale variation across object instances. Although these object detectors with feature pyramids achieve encouraging results, they have some limitations due to that they only simply construct the feature pyramid according to the inherent multiscale, pyramidal architecture of the backbones which are originally designed for object classification task. Newly, in this work, we present Multi-Level Feature Pyramid Network (MLFPN) to construct more effective feature pyramids for detecting objects of different scales. First, we fuse multi-level features (i.e. multiple layers) extracted by backbone as the base feature. Second, we feed the base feature into a block of alternating joint Thinned U-shape Modules and Feature Fusion Modules and exploit the decoder layers of each Ushape module as the features for detecting objects. Finally, we gather up the decoder layers with equivalent scales (sizes) to construct a feature pyramid for object detection, in which every feature map consists of the layers (features) from multiple levels. To evaluate the effectiveness of the proposed MLFPN, we design and train a powerful end-to-end one-stage object detector we call M2Det by integrating it into the architecture of SSD, and achieve better detection performance than state-of-the-art one-stage detectors. Specifically, on MSCOCO benchmark, M2Det achieves AP of 41.0 at speed of 11.8 FPS with single-scale inference strategy and AP of 44.2 with multi-scale inference strategy, which are the new stateof-the-art results among one-stage detectors. The code will be made available on https://github.com/qijiezhao/M2Det.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

EfficientDet: Scalable and Efficient Object Detection (2019, background): “Cross-scale connections are further studied in [13, 12, 34].”
NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection (2019, background): “[42] extends the idea to build stronger feature pyramid representations by employing multiple Ushape modules after a backbone model.”
MMDetection: Open MMLab Detection Toolbox and Benchmark (2019, background): “• M2Det [38]: a new feature pyramid network to construct more effective feature pyramids, proposed in 2018.”
Deep High-Resolution Representation Learning for Visual Recognition (2019, neutral): “0:9 FitnessNMS [140] ResNet-101 768 - 39:5 58:0 42:6 18:9 43:5 54:1 DetNet [86] DetNet59-FPN 800 2 40:3 62:1 43:8 23:6 42:6 50:0 CornerNet [79] Hourglass-104 511 - 40:5 56:5 43:1 19:4 42:7 53:9 M2Det [185] VGG16 800 ˘ 10 41:0 59:7 45:0 22:1 46:5 53:8 Faster R-CNN [92] ResNet-101-FPN 800 1 39:3 61:3 ”
YOLOv4: Optimal Speed and Accuracy of Object Detection (2020, methodology): “• Path-aggregation blocks: FPN [44], PAN [49], NAS-FPN [17], Fully-connected FPN, BiFPN [77], ASFF [48], SFAM [98]”
Feature Selective Anchor-Free Module for Single-Shot Object Detection (2019, background): “To achieve scale invariability, state-of-the-art detectors construct feature pyramids or multi-level feature towers [24, 8, 21, 22, 19, 38].”
CornerNet-Lite: Efficient Keypoint based Object Detection (2019, background): “On the other hand, one-stage detectors [34, 43, 9, 44, 31, 56, 23, 20, 60, 63, 48, 62] remove the region pooling step of two-stage detectors.”
Prime Sample Attention in Object Detection (2019, neutral): “Other variants include [26, 12, 27, 28].”
TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-captured Scenarios (2021, methodology): “Commonly used path-aggregation blocks in neck are: FPN [28], PANet [33], NAS-FPN [12], BiFPN [45], ASFF [32], SFAM [61].”
Deep Learning for Generic Object Detection: A Survey (2018, background): “Zhao ら [315] は,異なるスケールの物体の検出により効果的な特徴ピラミッドを構築するために,MultiLevel Feature Pyramid Network(MLFPN)を提案した.”
Dynamic Anchor Feature Selection for Single-Shot Object Detection (2019, methodology): “We also compare our method against some other excellent one-stage detectors (e.g., DES [37], CornerNet [17], M2Det [38]).”
Imbalance Problems in Object Detection: A Review (2019, background): “Different from Kong et al. [78] and Li et al. [79], MultiLevel FPN [80] stacks one highest and one lower level feature layers and recursively outputs a set of pyramidal features, which are all finally combined into a single feature pyramid in a scale-wise manner (see Figure 10(g)).”
DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution (2020, methodology): “There are two main categories of object detection methods: one-stage methods, e.g ., [45,50,56,60, 80, 81], and multi-stage methods, e.g ., [5, 7, 9, 25, 27, 58].”
Detecting tiny objects in aerial images: A normalized Wasserstein distance and a new benchmark (2022, neutral): “Recent advances for tiny object detection (TOD) mainly focus on improving the feature discrimination ability (Lin et al., 2017a; Zhao et al., 2019; Qiao et al., 2021; Li et al., 2017; Bai et al., 2018; Noh et al., 2019).”
Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism (2023, methodology): “M2Det [60] introduced an efficient MLFPN architecture with U-shape and Feature Fusion Modules.”
Towards Large-Scale Small Object Detection: Survey and Benchmarks (2022, background): “M2Det[89]constructsparallelbranchestodescribefeatures from shallow-to-deep in a cascade manner, where a Thinned U-shape Module is leveraged to capture more detailed information for small objects.”