Feature Pyramid Networks for Object Detection
By Tsung-Yi Lin, Piotr Doll'ar, Ross B. Girshick, Kaiming He, Bharath Hariharan, Serge J. Belongie. 2016
27,484 citations recorded by the source corpus.
Feature pyramids are a basic component in recognition systems for detecting objects at different scales. But pyramid representations have been avoided in recent object detectors that are based on deep convolutional networks, partially because they are slow to compute and memory intensive. In this paper, we exploit the inherent multi-scale, pyramidal hierarchy of deep convolutional networks to construct feature pyramids with marginal extra cost. A top-down architecture with lateral connections is developed for building high-level semantic feature maps at all scales. This architecture, called a Feature Pyramid Network (FPN), shows significant improvement as a generic feature extractor in several applications. Using a basic Faster R-CNN system, our method achieves state-of-the-art single-model results on the COCO detection benchmark without bells and whistles, surpassing all existing single-model entries including those from the COCO 2016 challenge winners. In addition, our method can run at 5 FPS on a GPU and thus is a practical and accurate solution to multi-scale object detection. Code will be made publicly available.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jianfeng Gao
- Quoc V. Le
- Mohit Bansal
- Dacheng Tao
- Yoshua Bengio
- Trevor Darrell
- Sergey Levine
- Alan L. Yuille
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Alan L. Yuille
- Yi Yang
Citations in the corpus
- End-to-End Object Detection with Transformers (2020, methodology): “This procedure of finding matching plays the same role as the heuristic assignment rules used to match proposal [37] or anchors [22] to ground truth objects in modern detectors.”
- Mask R-CNN (2017, methodology): “Images are resized such that their scale (shorter edge) is 800 pixels [27].”
- Momentum Contrast for Unsupervised Visual Representation Learning (2019, methodology): “, FPN [41]), which helps calibrate magnitudes.”
- Exploring the Limits of Weakly Supervised Pretraining (2018, methodology): “We use Mask R-CNN [4,36] and experiment with ResNeXt101 FPN [37] backbones of three different capacities (see Figure 8).”
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (2019, methodology): “Higher resolutions, such as 600x600, are also widely used in object detection ConvNets (He et al., 2017; Lin et al., 2017).”
- Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, background): “[71] append a light-weight DeepLab-inspired module [14] to the multi-scale features from FPN [58].”
- Attention Augmented Convolutional Networks (2019, methodology): “In the subsequent experiments, we test Attention Augmentation on standard computer vision architectures such as ResNets [14, 47, 13], and MnasNet [42] on the CIFAR100 [22], ImageNet [9] and COCO [25] datasets.”
- Weight Standardization (2019, methodology): “Detection and segmentation results on COCO [22] using Mask R-CNN [10] and FPN [23] with ResNet [12] and ResNeXt (32x4d) [44] as backbone.”
- Relation Networks for Object Detection (2017, methodology): “Most ablation experiments report detection accuracies on a subset of 5k unused val images (denoted as minival) as common practice [2, 32].”
- Non-local Neural Networks (2017, background): “The backbone is ResNet-101 with FPN [32].”
- VirTex: Learning Visual Representations from Textual Annotations (2020, methodology): “COCO Instance Segmentation: We train Mask RCNN [9] models with ResNet-50-FPN backbones [109].”
- All You Need Is Boundary: Toward Arbitrary-Shaped Text Spotting (2019, methodology): “2015), where the backbone is FPN (Lin et al. 2017) equipped with ResNet-50 (He et al.”
- Benchmarking Detection Transfer Learning with Vision Transformers (2021, methodology): “In this section we address two technical obstacles when using ViT as the backbone in Mask R-CNN: (1) how to adapt it to work with a feature pyramid network (FPN) [27] and (2) how to reduce its memory footprint and runtime to make benchmarking large ViT backbones tractable.”
- Unified Perceptual Parsing for Scene Understanding (2018, methodology): “[31] with a Pyramid Pooling Module (PPM) [16] appended on the last layer of the back-bone network before feeding it into the top-down branch in FPN. Top-right: We use features at various semantic levels.”
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour (2017, methodology): “As we will show in §5, we have found constant warmup particularly helpful for prototyping object detection and segmentation methods [9, 30, 25, 14] that fine-tune pre-trained layers together with newly initialized layers.”