T-CNN: Tubelets With Convolutional Neural Networks for Object Detection From Videos

By Kai Kang, Hongsheng Li, Junjie Yan, Xingyu Zeng, Binh Yang, Tong Xiao, Cong Zhang, Zhe Wang, Ruohui Wang, Xiaogang Wang and 1 more. 2016

526 citations recorded by the source corpus.

The state-of-the-art performance for object detection has been significantly improved over the past two years. Besides the introduction of powerful deep neural networks, such as GoogleNet and VGG, novel object detection frameworks, such as R-CNN and its successors, Fast R-CNN, and Faster R-CNN, play an essential role in improving the state of the art. Despite their effectiveness on still images, those frameworks are not specifically designed for object detection from videos. Temporal and contextual information of videos are not fully investigated and utilized. In this paper, we propose a deep learning framework that incorporates temporal and contextual information from tubelets obtained in videos, which dramatically improves the baseline performance of existing still-image detection frameworks when they are applied to videos. It is called T-CNN, i.e., tubelets with convolutional neueral networks. The proposed framework won newly introduced an object-detection-from-video task with provided data in the ImageNet Large-Scale Visual Recognition Challenge 2015. Code is publicly available at https://github.com/myfavouritekk/T-CNN.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild (2018, methodology): “Since visual tracking algorithms have been frequently used in a number of related areas such as video object detection [33], [34] and segmentation [35], multiple object tracking [36] and self-supervised learning [31], [32], keeping a compatible annotation standard encourages the development ofmore p”
NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naïve Bayes Data Fusion (2018, methodology): “of Cracks Sum of Scores Top-k (k = 4) Simple Bayes Naı̈ve Bayes [24] [4]”
Tracking Objects as Points (2020, background): “[16, 17] feed stacked consecutive frames into the network and do detection for a whole video segment.”
Object Detection in Video with Spatiotemporal Sampling Networks (2018, methodology): “Prior work [20, 21, 22, 23] has proposed to exploit such temporal information in videos by means of various post-processing steps aimed at making object detections coherent across time.”
Flow-Guided Feature Aggregation for Video Object Detection (2017, methodology): “Following the protocols in [18, 23], model training and evaluation are performed on the 3,862 video snippets from the training set and the 555 snippets from the validation set, respectively.”
Learning Spatial Regularization with Image-Level Supervisions for Multi-label Image Classification (2017, background): “Multi-label image classification is an important task in computer vision with various applications, such as scene recognition [4, 30, 31], multi-object recognition [25, 19, 18], human attribute recognition [24], etc.”
TransTrack: Multiple-Object Tracking with Transformer (2020, methodology): “T-CNN [16] feeds stacked consecutive frames into the network and performed detection for a whole video segment.”
Order-Free RNN with Visual Attention for Multi-Label Classification (2017, background): “…vision, a variety of applications benefit from the development and success of multi-label classification algorithms (Zhang and Zhou 2014; Boutell et al. 2004; Schapire and Singer 2000; Godbole and Sarawagi 2004; Lin et al. 2014; Kang et al. 2016b; 2016a; Boutell et al. 2004; Shao et al. 2016).”
Object Detection from Video Tubelets with Convolutional Neural Networks (2016, methodology): “Based on this work, a more advanced tubelet-based framework is further developed which won the ILSVRC2015 ImageNet VID challenge with provided data [15].”
Object Detection in 20 Years: A Survey (2019, unknown): “Object detection serves as a basis for many other computer vision tasks, such as instance segmentation [1], [2], [3], [4], image captioning [5], [6], [7], and object tracking [8].”
Temporal-Context Enhanced Detection of Heavily Occluded Pedestrians (2020, methodology): “Another direction is to associate proposal or detection boxes for tube classification and detection rescoring [34, 35, 33, 37, 32].”
A Comprehensive Survey of Scene Graphs: Generation and Application (2021, background): “On the other hand, as deep learning technology has developed, the corresponding object detection research has become increasingly mature [16], [62], [63], [64], [65], [66].”