BMN: Boundary-Matching Network for Temporal Action Proposal Generation
By Tianwei Lin, Xiao Liu, Xin Li, Errui Ding, Shilei Wen. 2019
726 citations recorded by the source corpus.
Temporal action proposal generation is an challenging and promising task which aims to locate temporal regions in real-world videos where action or event may occur. Current bottom-up proposal generation methods can generate proposals with precise boundary, but cannot efficiently generate adequately reliable confidence scores for retrieving proposals. To address these difficulties, we introduce the Boundary-Matching (BM) mechanism to evaluate confidence scores of densely distributed proposals, which denote a proposal as a matching pair of starting and ending boundaries and combine all densely distributed BM pairs into the BM confidence map. Based on BM mechanism, we propose an effective, efficient and end-to-end proposal generation method, named Boundary-Matching Network (BMN), which generates proposals with precise temporal boundaries as well as reliable confidence scores simultaneously. The two-branches of BMN are jointly trained in an unified framework. We conduct experiments on two challenging datasets: THUMOS-14 and ActivityNet-1.3, where BMN shows significant performance improvement with remarkable efficiency and generalizability. Further, combining with existing action classifier, BMN can achieve state-of-the-art temporal action detection performance.
Subfields: Computer Vision and Pattern Recognition, Time-Series & Forecasting
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Kristen Grauman
- Jitendra Malik
- Trevor Darrell
- Andrew Zisserman
- Cordelia Schmid
- Dacheng Tao
- Ming-Hsuan Yang
In Time-Series & Forecasting
- Chen Zhao
- Dima Damen
- Giovanni Maria Farinella
- Antonino Furnari
- Kristen Grauman
- Ali K. Thabet
- Yanghao Li
- Cordelia Schmid
Citations in the corpus
- Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “[140] Tianwei Lin, Xiao Liu, Xin Li, Errui Ding, and Shilei Wen.”
- Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100 (2020, methodology): “Action proposals are obtained using the public code of [79] then classified using SlowFast [51] (model trained as in Sec.”
- G-TAD: Sub-Graph Localization for Temporal Action Detection (2019, methodology): “5 IoU, G-TAD outperforms all one-stage detection methods, such as SS-TAD [4] and BMN [27].”
- Self-supervised Spatiotemporal Representation Learning by Exploiting Video Continuity (2021, methodology): “BMN is trained on the extracted features of the training set and then evaluated on the validation set.”
- ActionFormer: Localizing Moments of Actions with Transformers (2022, methodology): “Most previous works have considered using action proposals [36] or anchor windows [50], and developed convolutional [55, 85], recurrent [7], and graph [4, 76] neural networks for TAL.”
- Learning Salient Boundary Feature for Anchor-free Temporal Action Localization (2021, methodology): “To learn a more accurate boundary, we refer to former TAL methods [20, 18] indicating the importance of boundary or context feature.”
- InternVideo2: Scaling Foundation Models for Multimodal Video Understanding (2024, methodology): “We report mean Average Precision(mAP) under multiple tIoU as in [Lin et al., 2019, Chen et al., 2022a, Zhang et al., 2022, Yang et al., 2023].”
- Video Self-Stitching Graph Network for Temporal Action Localization (2020, methodology): “3 dataset [20, 19, 42, 2, 46], but since these methods are better suited for a small scale (e.g., 100 frames), most videos are actually down -scaled rather than up -scaled.”
- InternVideo: General Video Foundation Models via Generative and Discriminative Learning (2022, background): “Empirically, action recognition acts well as a meta task in video downstream applications, widely validated in [61, 62].”
- Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning (2023, background): “Dense video captioning lies at the intersection of event localization [25, 29, 33, 62, 65, 66, 84, 127] and event captioning [30,63,75,97,104].”