Soft-NMS — Improving Object Detection with One Line of Code

By Navaneeth Bodla, Bharat Singh, Ramalingam Chellappa, Larry S. Davis. 2017

2,100 citations recorded by the source corpus.

Non-maximum suppression is an integral part of the object detection pipeline. First, it sorts all detection boxes on the basis of their scores. The detection box M with the maximum score is selected and all other detection boxes with a significant overlap (using a pre-defined threshold) with M are suppressed. This process is recursively applied on the remaining boxes. As per the design of the algorithm, if an object lies within the predefined overlap threshold, it leads to a miss. To this end, we propose Soft-NMS, an algorithm which decays the detection scores of all other objects as a continuous function of their overlap with M. Hence, no object is eliminated in this process. Soft-NMS obtains consistent improvements for the coco-style mAP metric on standard datasets like PASCAL VOC2007 (1.7% for both R-FCN and Faster-RCNN) and MS-COCO (1.3% for R-FCN and 1.1% for Faster-RCNN) by just changing the NMS algorithm without any additional hyper-parameters. Using Deformable-RFCN, Soft-NMS improves state-of-the-art in object detection from 39.8% to 40.9% with a single model. Further, the computational complexity of Soft-NMS is the same as traditional NMS and hence it can be efficiently implemented. Since Soft-NMS does not require any extra training and is simple to implement, it can be easily integrated into any object detection pipeline. Code for Soft-NMS is publicly available on GitHub http://bit.ly/2nJLNMu.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

End-to-End Object Detection with Transformers (2020, background): “Learnable NMS methods [16,4] and relation networks [17] explicitly model relations between different predictions with attention.”
Relation Networks for Object Detection (2017, methodology): “Note that NMS and SoftNMS [4] methods are sequential and take about 5 ms on a CPU [4].”
Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection (2019, methodology): “As shown in Table 4, different values of k in [3, 5, 7, 9, 11, 13, 15, 17, 19] are used to train the detector.”
Cascade R-CNN: High Quality Object Detection and Instance Segmentation (2019, background): “The only detector that outperforms the Cascade R-CNN in Table 6 is SNIP [51], which uses multi-scale training and inference, a larger input size, a stronger backbone, Soft NMS, and some other enhancements.”
Objects as Points (2019, methodology): “We use the key-point values ˆ Y x i y i c as a measure of its detection conﬁdence, and produce a bounding box at location [2] object size [2] 3D size [3] depth [1] orientation [8] joint locations where ( δ ˆ x i , δ ˆ y i ) = ˆ O ˆ x i , ˆ y i is the offset prediction and ( ˆ w i , ˆ h i ) = ˆ S ˆ…”
Pix2seq: A Language Modeling Framework for Object Detection (2021, background): “Non-maximum suppression (Bodla et al., 2017) is often required to prevent duplicate predictions.”
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows (2021, methodology): “For system-level comparison, we adopt an improved HTC [9] (denoted as HTC++) with instaboost [20], stronger multi-scale training [7], 6x schedule (72 epochs), soft-NMS [5], and ImageNet-22K pre-trained model as initialization.”
Exploring Plain Vision Transformer Backbones for Object Detection (2022, methodology): “For a fairer comparison, here we make two changes following our competitors: we adopt soft-nms [3] as is used by all competitors [40,32,34,39] in this table and increase the input size (from 1024 to 1280) following [34,39].”
Hybrid Task Cascade for Instance Segmentation (2019, methodology): “We apply SoftNMS [3] to box results.”
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, background): “Each level of this proxy tree involves manually-designed modules, such as anchors [77], box assignment rules [105], non-maximum suppression (NMS) [7], thing-stuff merging [98], etc.”
CornerNet: Detecting Objects as Paired Keypoints (2018, methodology): “2 - - - - - Soft-NMS [2] Aligned-Inception-ResNet 40.”
MMDetection: Open MMLab Detection Toolbox and Benchmark (2019, background): “• Soft NMS [1]: an alternative to NMS, proposed in 2017.”
Rethinking Pre-training and Self-training (2020, methodology): “3 All SpineNet models also use Soft-NMS with a sigma of 0.3 [57].”
Acquisition of Localization Confidence for Accurate Object Detection (2018, methodology): “Quantitative results show that our method outperforms traditional NMS and other variants such as Soft-NMS [2].”
Single-Shot Refinement Neural Network for Object Detection (2017, methodology): “4% AP, exceeding most of the detection methods except Faster RCNN w TDM [42], Deformable R-FCN [6], RetinaNet800 [27], umd det [2], and G-RMI [21].”
Mask Scoring R-CNN (2019, methodology): “k = 100) scoring boxes after SoftNMS [2] are selected.”
Efficient Attention: Attention with Linear Complexities (2018, methodology): “ with other state-of-the-art methods on the MS-COCO 2017 dataset. EA Mask R-CNN with a ResNeXt-101 Model Backbone AP TDM Faster R-CNN [22] Inc.-ResNet-v2 36:8 Mask R-CNN [9] ResNeXt-101 39:8 Soft-NMS [2] Algn.-Inc.-ResNet 40:9 LH R-CNN [13] ResNet-101 41:5 Fitness NMS [24] ResNet-101 41:8 EA Mask R-”
CenterNet: Keypoint Triplets for Object Detection (2019, methodology): “2 - - - - - Soft-NMS [2] Aligned-Inception-ResNet ∼ 1300×800 ∼ 1300×800 40.”