Deep Feature Pyramid Reconfiguration for Object Detection

By Tao Kong, Fuchun Sun, Wen-bing Huang, Huaping Liu. 2018

218 citations recorded by the source corpus.

State-of-the-art object detectors usually learn multi-scale representations to get better results by employing feature pyramids. However, the current designs for feature pyramids are still inefficient to integrate the semantic information over different scales. In this paper, we begin by investigating current feature pyramids solutions, and then reformulate the feature pyramid construction as the feature reconfiguration process. Finally, we propose a novel reconfiguration architecture to combine low-level representations with high-level semantic features in a highly-nonlinear yet efficient way. In particular, our architecture which consists of global attention and local reconfigurations, is able to gather task-oriented features across different spatial locations and scales, globally and locally. Both the global attention and local reconfiguration are lightweight, in-place, and end-to-end trainable. Using this method in the basic SSD system, our models achieve consistent and significant boosts compared with the original model and its other variations, without losing real-time processing speed.

Subfields: Computer Vision and Pattern Recognition, Neurology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Neurology

Citations in the corpus

Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection (2019, background): “Thereafter, plenty of works are presented to boost its performance in different aspects, such as fusing context information from different layers [24, 12, 69], training from scratch [50, 73], introducing new loss function [33, 6], anchor refinement and matching [66, 67], architecture redesign [21, 2”
Deformable DETR: Deformable Transformers for End-to-End Object Detection (2020, background): “PANet (Liu et al., 2018b) further adds an bottom-up path on the top of FPN. Kong et al. (2018) combines features from all scales by a global attention operation.”
EfficientDet: Scalable and Efficient Object Detection (2019, background): “Cross-scale connections are further studied in [13, 12, 34].”
NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection (2019, background): “Many recent works [7, 16, 17, 34, 38, 39, 40, 43, 41] propose various crossscale connections or operations to combine features to gen-”
Libra R-CNN: Towards Balanced Learning for Object Detection (2019, background): “[17] proposed a novel efficient pyramid based on SSD that integrates the features in a highly-nonlinear yet efficient way.”
Efficient Neural Architecture Transformation Searchin Channel-Level for Object Detection (2019, background): “Many recent works [15, 14, 41] propose various multi-scale integration strategies to generate pyramidal feature representations.”
High-Resolution Representations for Labeling Pixels and Regions (2019, unknown): “DFPR [48] ResNet-101 512 1× 34.”
Learning Spatial Fusion for Single-Shot Object Detection (2019, background): “Kong et al. [15] aggregate feature maps at all scales to a speciﬁc scale and then produce features at each scale by a global attention operation on the combined features.”
CARAFE: Content-Aware ReAssembly of FEatures (2019, background): “[18, 21, 15, 41, 28] exploits multi-scale feature pyramids to deal with objects at different scales.”
Localization Distillation for Object Detection (2022, background): “Object localization is a fundamental issue in object detection [18], [33], [44], [64], [65], [73], [74], [75], [80], [93].”
Recent Advances in Deep Learning for Object Detection (2019, unknown): “9 FPN-Reconfig [115] ResNet-101 2018 ∼ 600× 1000 82.”
You Only Look One-level Feature (2021, methodology): “line of studies of designing complex fusion methods manually [22, 14, 25], or via Neural Architecture Search (NAS) algorithms [7, 34].”
ScratchDet: Exploring to Train Single-Shot Object Detectors from Scratch (2018, unknown): “2 DFPR300 [17] trainval VGG-16 28.”
High-Level Semantic Feature Detection: A New Perspective for Pedestrian Detection (2019, background): “Note that more complicated feature fusion strategies like [23, 15, 17] can be explored to further improve the detection performance, but it is not in the scope of this work.”
Effective Fusion Factor in FPN for Tiny Object Detection (2020, methodology): “Kong [13] proposed the method of global attention and local reconﬁguration, which combined the high-level semantic features with the low-level representation to reconstruct the feature pyramid.”