Automatic annotation of human actions in video

By Olivier Duchenne, Ivan Laptev, Josef Sivic, Francis R. Bach, Jean Ponce. 2009

316 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

End-to-End Learning of Visual Representations From Uncurated Instructional Videos (2019, neutral): “Multiple instance learning methods have been employed in many weakly-supervised video understanding problems including: person recognition in movies using scripts [12, 50, 61], anomaly detection [70], weakly supervised action classification [47, 68] and localization [17, 22, 82], co-reference resolu”
Unsupervised Learning from Narrated Instruction Videos (2015, background): “Particularly related to ours is the work [4] which explores the known order of events to localize and learn actions in training data.”
Dense-Captioning Events in Videos (2017, methodology): “To enable action localization, temporal action proposal methods started from traditional sliding window approaches [7] and later started building models to propose a handful of possible action segments [8, 5].”
Movie Description (2016, methodology): “Movie scripts have been used for automatic discovery and annotation of scenes and human actions in videos (Duchenne et al. 2009; Laptev et al. 2008; Marszalek et al. 2009), as well as a resource to construct activity knowledge base (Tandon et al. 2015; de Melo and Tandon 2016).”
A dataset for Movie Description (2015, methodology): “Second, we propose to use semantic parsing of the sentence to extract training labels for recently proposed translation approach [56] for video description.”
Weakly Supervised Action Labeling in Videos under Ordering Constraints (2014, methodology): “This type of weak supervisory signal has been used before in classification [22] and temporal localization [7] tasks.”
Unsupervised Semantic Parsing of Video Collections (2015, methodology): “Additionally, several method for localizing instances of actions in rather longer video sequences have been developed [10, 18, 34, 6, 47].”
Connectionist Temporal Modeling for Weakly Supervised Action Labeling (2016, background): “Using the ordering constraint, OCDC extends previous work [9] to deal with multiple actions in a video.”
Large-Scale Weakly-Supervised Pre-Training for Video Action Recognition (2019, background): “[8, 21, 23, 47] analyzed movie scripts for automatic annotation of human actions for recognizing and localizing actions and actors.”
Towards Weakly-Supervised Action Localization (2016, methodology): “[27] use a discriminative clustering on short video segments to identify the localization in the training set and learn a classiﬁer.”
Anticipating Human Activities Using Object Affordances for Reactive Robotic Response (2013, methodology): “are obtained automatically using weak-supervision [35] or by hand [36].”
NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learning (2018, background): “[5] first addressed the problem of segmenting actions within videos, assuming that a clip contains not only frames of an action but also background frames.”
Action Search: Spotting Actions in Videos and Its Application to Temporal Action Localization (2017, methodology): “Early methods have relied on the sliding-window-plus-classifier combination to produce the actions temporal boundaries [13,15,23,29].”
Visual event recognition in videos by learning from web data (2010, methodology): “In order to locate the actions from movies, a new discriminative clustering algorithm [11] was developed based on the weakly-labeled training data that can be readily obtained from movie scripts without any cost of manual annotation.”
A flexible model for training action localization with varying levels of supervision (2018, background): “Prior work has explored the use of readily-available sources of information such as movie scripts [4, 8] to discover actions in clips using text analysis.”
Temporal Localization of Actions with Actoms (2013, unknown): “Similar to [ 8], we use the k -means algorithm with a number of clusters set to v = 1000 for our localization experiments, while, similar to [49], we use v = 4000 for our classiﬁcation-by-localization experiments.”
Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-Supervised Object and Action Localization (2017, background): “Weakly-supervised approaches learn from movie scripts [29, 12] or an ordered list of actions [5, 18].”
MovieNet: A Holistic Dataset for Movie Understanding (2020, background): “genre [89,63], character [1,3,31,48,65,22,35], action [39,21,46,5,6], scene [53,11,30,49,15,51] and description [61].”
Modeling the Temporal Extent of Actions (2010, methodology): “Unlike the authors of [4], who strive to generate croppings which perform as well as human-labeled data, we consider the performance of a classifier trained on manually cropped actions to be a baseline.”
Weakly supervised learning of actions from transcripts (2016, background): “[1] also use an automatic script alignment to provide coarse temporal localization of human actions and point out that the textual based detection usually leads to temporal misalignment.”