SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization

By Xianzhi Du, Tsung-Yi Lin, Pengchong Jin, Golnaz Ghiasi, Mingxing Tan, Yin Cui, Quoc V. Le, Xiaodan Song. 2019

184 citations recorded by the source corpus.

Convolutional neural networks typically encode an input image into a series of intermediate features with decreasing resolutions. While this structure is suited to classification tasks, it does not perform well for tasks requiring simultaneous recognition and localization (e.g., object detection). The encoder-decoder architectures are proposed to resolve this by applying a decoder network onto a backbone model designed for classification tasks. In this paper, we argue encoder-decoder architecture is ineffective in generating strong multi-scale features because of the scale-decreased backbone. We propose SpineNet, a backbone with scale-permuted intermediate features and cross-scale connections that is learned on an object detection task by Neural Architecture Search. Using similar building blocks, SpineNet models outperform ResNet-FPN models by 3%+ AP at various scales while using 10-20% fewer FLOPs. In particular, SpineNet-190 achieves 52.1% AP on COCO, attaining the new state-of-the-art performance for single model object detection without test-time augmentation. SpineNet can transfer to classification tasks, achieving 5% top-1 accuracy improvement on a challenging iNaturalist fine-grained dataset. Code is at: https://github.com/tensorflow/tpu/tree/master/models/official/detection.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation (2020, methodology): “0 [57, 11] and longer training significantly improves performance (see Figure 4, right plot).”
Bottleneck Transformers for Visual Recognition (2021, background): “One way to address this is to increase the amount of multi-scale jitter which has been known to improve the performance of detection and segmentation systems [16, 18].”
Dynamic Head: Unifying Object Detection Heads with Attentions (2021, methodology): “Our method is competitive and more efﬁcient to learn compared with EffcientDet [27] and SpineNet [8], with a signiﬁcantly less 1 / 20 training time.”
CBNet: A Composite Backbone Network Architecture for Object Detection (2021, background): “6% AP, surpassing other anchor-based detectors [26], [27], [38], [39], [40], [41], [66], [80].”
Simple Training Strategies and Model Scaling for Object Detection (2021, unknown): “The same scale jittering strategy is used and studied in [8, 36, 9].”
A Simple Semi-Supervised Learning Framework for Object Detection (2020, methodology): “Popular object detection frameworks include Region-based CNN (RCNN) [15,14,41,17,28], YOLO [39,40], SSD [31], etc [26,51,11].”
Scaled-YOLOv4: Scaling Cross Stage Partial Network (2020, methodology): “In addition, recently there are NAS and model scaling methods speciﬁcally proposed for object detection, such as SpineNet [5] and EfﬁcientDet [31].”