Seq-NMS for Video Object Detection

By Wei Han, Pooya Khorrami, T. Paine, Prajit Ramachandran, Mohammad Babaeizadeh, Humphrey Shi, Jianan Li, Shuicheng Yan, Thomas S. Huang. 2016

323 citations recorded by the source corpus.

Video object detection is challenging because objects that are easily detected in one frame may be difficult to detect in another frame within the same clip. Recently, there have been major advances for doing object detection in a single image. These methods typically contain three phases: (i) object proposal generation (ii) object classification and (iii) post-processing. We propose a modification of the post-processing phase that uses high-scoring object detections from nearby frames to boost scores of weaker detections within the same clip. We show that our method obtains superior results to state-of-the-art single image object detection techniques. Our method placed 3rd in the video object detection (VID) task of the ImageNet Large Scale Visual Recognition Challenge 2015 (ILSVRC2015).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Do Image Classifiers Generalize Across Time? (2019, neutral): “-Viddataset [29]. WeﬁrstprovidebackgroundonImageNet-Vidandthendescribehowweusedittoconstruct ImageNet-Vid-Robust. 2.1 ImageNet-Vid The2015ImageNet-Viddatasetiswidelyusedfortrainingvideoobjectdetectors[14]aswellas trackers[3]. Wechosetoworkwiththe2017ImageNet-Viddatasetbecauseitisasupersetofthe 2015d”
Video Instance Segmentation (2019, methodology): “This is an offline algorithm following SeqNMS [10].”
FairMOT: On the Fairness of Detection and Re-identification in Multiple Object Tracking (2020, result): “Similar ideas have also been explored in Han et al. (2016), Kang et al. (2016, 2017), Tang et al. (2019), Pang et al. (2020).”
T-CNN: Tubelets With Convolutional Neural Networks for Object Detection From Videos (2016, methodology): “[20] proposed a sequence NMS method to associated still-image detections into sequences and apply the sequence-level NMS on the results.”
Flow-Guided Feature Aggregation for Video Object Detection (2017, methodology): “In particular, we tested three prevalent techniques, namely, motion guided propagation (MGP) [18], Tubelet rescoring [18], and Seq-NMS [12].”
Object Detection in Video with Spatiotemporal Sampling Networks (2018, methodology): “Prior work [20, 21, 22, 23] has proposed to exploit such temporal information in videos by means of various post-processing steps aimed at making object detections coherent across time.”
STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos (2020, methodology): “[94] adapted several existing methods [5,26,85,80,95] for the VIS task, and also proposed their own method (MaskTrack-RCNN) which extends Mask R-CNN with additional cues for data association.”
A systematic framework for natural perturbations from videos (2019, unknown): “-Viddataset [29]. WeﬁrstprovidebackgroundonImageNet-Vidandthendescribehowweusedittoconstruct ImageNet-Vid-Robust. 2.1 ImageNet-Vid The2015ImageNet-Viddatasetiswidelyusedfortrainingvideoobjectdetectors[14]aswellas trackers[3]. Wechosetoworkwiththe2017ImageNet-Viddatasetbecauseitisasupersetofthe 2015d”
Crossover Learning for Fast Online Video Instance Segmentation (2021, methodology): “(4) Post-processing, e.g ., Seq-NMS [24] and ObjLink [48] reﬁne video object detection results based on dynamic programming and learnable object tubelet linking, respectively.”
Video Object Detection with an Aligned Spatial-Temporal Memory (2017, methodology): “Therefore, in recent years, there has been a growing interest in the community on designing video object detectors [18,19,29,25,14,51,50].”
Temporal-Channel Transformer for 3D Lidar-Based Video Object Detection in Autonomous Driving (2020, background): “Previous works on video object detection [11, 14, 3, 20] has demonstrated that exploring spatial and temporal correlations is useful for learning an accurate representation for the current frame and achieving better object detection performance.”
Object Detection in Videos with Tubelet Proposal Networks (2017, methodology): “Existing methods [14, 7, 13] mainly uses temporal information as a post-processing method, either propagating detections to neighboring frames or temporally smoothing detection scores.”
Temporal-Context Enhanced Detection of Heavily Occluded Pedestrians (2020, methodology): “Another direction is to associate proposal or detection boxes for tube classification and detection rescoring [34, 35, 33, 37, 32].”
Memory Enhanced Global-Local Aggregation for Video Object Detection (2020, methodology): “The main focus of recent methods [16, 12, 37, 36, 35, 9, 27, 1, 31, 7, 30] towards solving video object detection is improving the performance of per-frame detection by exploiting information in the temporal dimension.”
A Survey of Deep Learning-Based Object Detection (2019, background): “Video detectors [280]–[289] exploit temporal contexts to”