Temporal Context Network for Activity Localization in Videos

By Xiyang Dai, Bharat Singh, Guyue Zhang, Larry S. Davis, Yan Qiu Chen. 2017

257 citations recorded by the source corpus.

We present a Temporal Context Network (TCN) for precise temporal localization of human activities. Similar to the Faster-RCNN architecture, proposals are placed at equal intervals in a video which span multiple temporal scales. We propose a novel representation for ranking these proposals. Since pooling features only inside a segment is not sufficient to predict activity boundaries, we construct a representation which explicitly captures context around a proposal for ranking it. For each temporal segment inside a proposal, features are uniformly sampled at a pair of scales and are input to a temporal convolutional neural network for classification. After ranking proposals, non-maximum suppression is applied and classification is performed to obtain final detections. TCN outperforms state-of-the-art methods on the ActivityNet dataset and the THU-MOS14 dataset.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Skeleton-Based Online Action Prediction Using Scale Selection Network (2019, methodology): “Beside the online action prediction task, the problem of temporal action detection [22], [56], [56], [57], [58], [59], [60], [61], [62], [63],”
SSNet: Scale Selection Network for Online 3D Action Prediction (2018, background): “Beside the online action prediction task, the problem of temporal action detection [68, 73, 48, 33, 7, 53, 68, 14, 54, 16, 15, 65, 82] also copes with untrimmed videos.”
Graph Convolutional Networks for Temporal Action Localization (2019, methodology): “As already demonstrated in [8, 6], exploring the contextual information is of great help in refining the detection boundary and increasing the detection accuracy eventually.”
MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment (2018, background): “To better localize temporal boundaries, a large body of work incorporated deep networks into the detection framework and obtained improved performance [5, 28, 10, 41, 2, 61, 42, 53, 59].”
Temporal Action Detection with Structured Segment Networks (2017, unknown): “2017), and temporal context network (Dai et al. 2017).”
G-TAD: Sub-Graph Localization for Temporal Action Detection (2019, background): “Therefore, incorporating context into temporal action detection has become an important strategy to boost detection accuracy in the recent literature [11, 15, 9, 33, 44, 56, 30].”
Action Search: Spotting Actions in Videos and Its Application to Temporal Action Localization (2017, unknown): “ 0:3 0:4 0:5 0:6 0:7 S Frame Glimpses [44] 36:0 26:4 17:1 { { 40 Shou et al. [35] 36:3 28:7 19:0 { { 100 Shou et al. [34] 40:1 29:4 23:3 13:1 7:9 100 Gao et al. [17] 44:1 34:9 25:6 { { 100 Dai et al. [11] { 33:3 25:6 15:9 9:0 100 Xu et al. [43] 44:8 35:6 28:9 { { 100 Buch et al. [3] 45:7 { 29:2 { 9:”
One-Shot Action Localization by Learning Sequence Matching Network (2018, background): “Action localization Fully supervised action localization problem has been extensively studied [25, 30, 38, 49, 35, 48, 15, 50, 51, 34, 23, 14, 28, 47, 8].”
Horizontal Pyramid Matching for Person Re-identification (2018, methodology): “In the future, we are plan to simultaneously optimize Re-ID and other related tasks, such as human activity recognition (Dai et al. 2017a; Dai et al. 2017b; Dai et al. 2019).”
BMN: Boundary-Matching Network for Temporal Action Proposal Generation (2019, unknown): “Method [7] [13] [19] [10] [17] BMN”
Weakly-Supervised Action Localization With Background Modeling (2019, background): “More recent state-of-the-art methods [11, 10, 32, 5, 4] borrow intuitions from the recent object detection frameworks (e.”
Diagnosing Error in Temporal Action Detectors (2018, background): “For example, multiple works have tried to capture the temporal context along the video [8,12,24,35,45] as a proxy for the localization of the actions.”
Rethinking the Faster R-CNN Architecture for Temporal Action Localization (2018, methodology): “used two-stream features, but either did not perform fusion [16] or only tried the early fusion scheme [9, 15].”