W-TALC: Weakly-supervised Temporal Activity Localization and Classification

By Sujoy Paul, Sourya Roy, Amit K. Roy-Chowdhury. European Conference on Computer Vision, 2018

340 citations recorded by the source corpus.

Most activity localization methods in the literature suffer from the burden of frame-wise annotation requirement. Learning from weak labels may be a potential solution towards reducing such manual labeling effort. Recent years have witnessed a substantial influx of tagged videos on the Internet, which can serve as a rich source of weakly-supervised training data. Specifically, the correlations between videos with similar tags can be utilized to temporally localize the activities. Towards this goal, we present W-TALC, a Weakly-supervised Temporal Activity Localization and Classification framework using only video-level labels. The proposed network can be divided into two sub-networks, namely the Two-Stream based feature extractor network and a weakly-supervised module, which we learn by optimizing two complimentary loss functions. Qualitative and quantitative results on two challenging datasets - Thumos14 and ActivityNet1.2, demonstrate that the proposed method is able to detect activities at a fine granularity and achieve better performance than current state-of-the-art methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/b2ed766ca48d42ac57e16f30ca039fc8aa960189

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision (2020, methodology): “Following the principles of MIL [36,27], we use the average of K-max activation ( C and C) over the temporal dimension rather than the whole activations to compute y and y , and K is defined as ⌊ T ′ q + 1 ⌋ .”
Weakly-Supervised Action Localization With Background Modeling (2019, methodology): “W-TALC [21] introduces a system with k-max Multiple Instance Learning and explicitly identifies the correlations between videos of similar categories by a co-activity similar loss.”
Completeness Modeling and Context Separation for Weakly Supervised Temporal Action Localization (2019, methodology): “Most of existing weakly supervised methods [45, 33, 38, 36, 52] fall into the framework of Multiple Instance Learning (MIL) [54].”
Action Recognition From Single Timestamp Supervision in Untrimmed Videos (2019, methodology): “We choose Untrimmed Net [32] amongst the aforementioned works [32, 22, 28, 23], which is used to extract features in [23] and is the backbone model of [28], due to the availability of published code.”
HANet: Hierarchical Alignment Networks for Video-Text Retrieval (2021, neutral): “To this end, we introduce a multiple instance learning (MIL) based mechanism inspired by [32, 43, 44], which can be presented as follows,”
End-to-End Temporal Action Detection With Transformer (2021, methodology): “There are also some weakly supervised methods [27, 21, 30, 31, 34, 40, 46], which are beyond the scope of this paper.”
ActionSpotter: Deep Reinforcement Learning Framework for Temporal Action Spotting in Videos (2020, background): “This setting allows good reproducibility as such features are provided by [44] and [37].”
Action Modifiers: Learning From Adverbs in Instructional Videos (2019, methodology): “We use published code of two state-of-the-art weakly supervised action localization methods: W-TALC [40] and CMCS [27].”