SNIPER: Efficient Multi-Scale Training

By Bharat Singh, Mahyar Najibi, Larry S. Davis. 2018

550 citations recorded by the source corpus.

We present SNIPER, an algorithm for performing efficient multi-scale training in instance level visual recognition tasks. Instead of processing every pixel in an image pyramid, SNIPER processes context regions around ground-truth instances (referred to as chips) at the appropriate scale. For background sampling, these context-regions are generated using proposals extracted from a region proposal network trained with a short learning schedule. Hence, the number of chips generated per image during training adaptively changes based on the scene complexity. SNIPER only processes 30% more pixels compared to the commonly used single scale training at 800x1333 pixels on the COCO dataset. But, it also observes samples from extreme resolutions of the image pyramid, like 1400x2000 pixels. As SNIPER operates on resampled low resolution chips (512x512 pixels), it can have a batch size as large as 20 on a single GPU even with a ResNet-101 backbone. Therefore it can benefit from batch-normalization during training without the need for synchronizing batch-normalization statistics across GPUs. SNIPER brings training of instance level recognition tasks like object detection closer to the protocol for image classification and suggests that the commonly accepted guideline that it is important to train on high resolution images for instance level visual recognition tasks might not be correct. Our implementation based on Faster-RCNN with a ResNet-101 backbone obtains an mAP of 47.6% on the COCO dataset for bounding box detection and can process 5 images per second during inference with a single GPU. Code is available at this https URL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Efficiency & Compression

Citations in the corpus

Objects as Points (2019, background): “Sophisticated two-stage detectors [31,35,48,63] are more accurate, but also slower.”
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows (2021, background): “Unlike the word tokens that serve as the basic elements of processing in language Transformers, visual elements can vary substantially in scale, a problem that receives attention in tasks such as object detection [38, 49, 50].”
Deep High-Resolution Representation Learning for Visual Recognition (2019, background): “Table 11 reports the comparison of our network to state-of-the-art single-model object detectors on COCO test-dev without using multi-scale training and multi-scale testing that are done in [85], [97], [110], [115], [127], [128].”
Scale-Aware Trident Networks for Object Detection (2019, methodology): “Similar to SNIP [38], we define a valid range [li, ui] for each branch i.”
Libra R-CNN: Towards Balanced Learning for Object Detection (2019, methodology): “Other methods focus on cascade procedures [24], duplicate removal [14, 13], multi-scales [2, 1, 31, 30], adversarial learning [37] and more contextual [36].”
Objects365: A Large-Scale, High-Quality Dataset for Object Detection (2019, methodology): “Deformable ConvNet [4, 39], Soft Sampling [34], SNIP [32], SNIPER [33], and Cascade R-CNN [1], DetNet [21] are introduced to further improve the performance.”
Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis (2021, methodology): “To avoid quadratic overflow of computing self-attention at scales [29, 41], Swin Transformer [30, 31] is proposed to construct hierarchical encoding by a shiftedwindow mechanism.”
You Only Look One-level Feature (2021, methodology): “In CNN-based detectors, the image pyramids method also wins some researchers’ [31, 32] praise as it can achieve higher performance out of the box.”
CornerNet-Lite: Efficient Keypoint based Object Detection (2019, methodology): “AutoFocus [37], which builds upon SNIPER [52] that improved R-CNN training, adds a branch to Faster R-CNN to predict the regions that are likely to contain small objects.”
Effective Fusion Factor in FPN for Tiny Object Detection (2020, methodology): “SNIP [25] and SNIPER [26] use a scale regularization strategy to guarantee the size of the object is in a ﬁxed range for different resolution images.”
Explicit Shape Encoding for Real-Time Instance Segmentation (2019, background): “Besides, the works on augmenting the performance of instance segmentation frameworks through data augmentation [7, 36], scale normalization [33] can be easily integrated to our system.”
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks (2019, background): “In recent years, we have witnessed the remarkable achievements of CNNs in many computer vision tasks [40, 48, 37, 51, 24].”
Object Detection in 20 Years: A Survey (2019, unknown): “Some researchers have further proposed a more efficient training strategy: SNIP with Efficient Resampling (SNIPER) [177], i.”
Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors (2019, background): “We briefly review the inner workings of object detectors [20,4,29,17], most of which can be described as two-stage frameworks (e.”
Recent Advances in Deep Learning for Object Detection (2019, methodology): “A more intensive data augmentation strategy is used in one-stage detectors including rotation, random crops, expanding and color jittering [42, 106, 146].”