Completeness Modeling and Context Separation for Weakly Supervised Temporal Action Localization
By Daochang Liu, Tingting Jiang, Yizhou Wang. 2019
237 citations recorded by the source corpus.
Temporal action localization is crucial for understanding untrimmed videos. In this work, we first identify two underexplored problems posed by the weak supervision for temporal action localization, namely action completeness modeling and action-context separation. Then by presenting a novel network architecture and its training strategy, the two problems are explicitly looked into. Specifically, to model the completeness of actions, we propose a multi-branch neural network in which branches are enforced to discover distinctive action parts. Complete actions can be therefore localized by fusing activations from different branches. And to separate action instances from their surrounding context, we generate hard negative data for training using the prior that motionless video clips are unlikely to be actions. Experiments performed on datasets THUMOS'14 and ActivityNet show that our framework outperforms state-of-the-art methods. In particular, the average mAP on ActivityNet v1.2 is significantly improved from 18.0% to 22.4%. Our code will be released soon.
Subfields: Computer Vision and Pattern Recognition, Information Extraction
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Giovanni Maria Farinella
- Antonino Furnari
- Kristen Grauman
- Dima Damen
- Jitendra Malik
- Luc Van Gool
- Andrew Zisserman
- Cordelia Schmid
In Information Extraction
- Ivan Laptev
- Dima Damen
- Giovanni Maria Farinella
- Antonino Furnari
- Jianxin Wu
- Chen-Lin Zhang
- Hazel Doughty
- Yin Li
Citations in the corpus
- Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100 (2020, methodology): “ervision. Contrary to Sec. 4.1, this challenge does not use action start/end times during training. Previous weakly-supervised approaches utilised video-level or transcript supervision, where the set [64,65,66,67,68,69] or sequence [70,71,72,73,74,75] of actions in the video are used in training, wi”
- Action Modifiers: Learning From Adverbs in Instructional Videos (2019, methodology): “We use published code of two state-of-the-art weakly supervised action localization methods: W-TALC [40] and CMCS [27].”
- ActionFormer: Localizing Moments of Actions with Transformers (2022, methodology): “We used two-stream I3D [12] pretrained on Kinetics to extract the video features on THUMOS14, following [41, 84].”