Natural Language Description of Human Activities from Video Images Based on Concept Hierarchy of Actions

By A. Kojima, Takeshi Tamura, K. Fukunaga. 2002

372 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Geography, Planning and Development

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Geography, Planning and Development

Citations in the corpus

Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks (2015, neutral): “ing human activities with natural language [23].”
A Dataset and Exploration of Models for Understanding Video Data through Fill-in-the-Blank Question-Answering (2016, background): “Early models tackling video captioning such as [17, 32], focused on constrained domains with limited appearance of activities and objects in videos and depended heavily on hand-crafted video features, followed by a template-based or shallow statistical machine translation.”
Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation (2017, background): “There is a growing body of work that generalises the task from static inputs to sequential ones, especially videos (e.g. Kojima et al., 2002; Regneri et al., 2013; Venugopalan et al., 2015b, 2015a).”
MSR-VTT: A Large Video Description Dataset for Bridging Video and Language (2016, background): “The first research dimension applies video representation to template-based or statistical machine translations [2, 15, 28].”
Spatio-Temporal Dynamics and Semantic Attribute Enriched Visual Encoding for Video Captioning (2019, background): “[26] that focuses on describing videos of one person performing one action only.”
A dataset for Movie Description (2015, background): “In recent years there has been an increased interest in automatically describing images [23, 39, 40, 50, 45, 40, 41, 34, 61, 22] and videos [37, 27, 8, 28, 32, 62, 16, 26, 64, 55] with natural language.”
Towards textually describing complex video contents with audio-visual concept classifiers (2011, methodology): “In [3], an interesting method was proposed for textually describing videos.”
A survey of vision-based methods for action representation, segmentation and recognition (2011, methodology): “Other dynamic models that have been used for action recognition are: auto regressive models [13,98,4], time delayed neural networks [146], context-free grammars [48] and feature-structure grammars [58,124].”
Movie Description (2016, background): “In the past video description has been addressed in controlled settings (Barbu et al. 2012; Kojima et al. 2002), on a small scale (Das et al. 2013; Guadarrama et al. 2013; Thomason et al. 2014) or in single domains like cooking (Rohrbach et al. 2014, 2013; Don-ahue et al. 2015).”
End-to-end Generative Pretraining for Multimodal Video Captioning (2022, methodology): “Early works in video captioning consisted of rule-based methods [10,22], where subjects, verbs and objects (SVO-triplets) detected from the video were combined into sentence templates.”
Visual to Text: Survey of Image and Video Captioning (2019, background): “who constructed an action concept hierarchy represented by a hierarchical case frame [28].”
SAVE: A framework for semantic annotation of visual events (2008, methodology): “Compared to [ 9 ], we use grammarbased approaches for inferring and annotating a broader”
Automated Textual Descriptions for a Wide Range of Video Events with 48 Human Actions (2012, background): “…textual descriptions from video has been focused on: – using speech recognition to generate video subscriptions [7], – detecting and extracting text which is present in the video [7], – detecting patterns in a restricted environment and use the detected patterns to generate a description [8,9].”
Baby talk: Understanding and generating simple image descriptions (2011, background): “In addition to these generation efforts in images, there has also been work related to linking humans and their actions in video [21], [23], [24].”
BabyTalk: Understanding and Generating Simple Image Descriptions (2013, background): “In addition to these generation efforts in images, there has also been work related to linking humans and their actions in video [21], [23], [24].”
How Many Words Is a Picture Worth? Automatic Caption Generation for News Images (2010, background): “Kojima et al. (2002, 2008) create descriptions for human activities in ofﬁce scenes.”
Automatic Caption Generation for News Images (2013, background): “Content selection makes use of dictionaries that specify a mapping between words and image regions or features [10], [11], [12], [13], [14], and surface realization uses human written templates IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 35, NO. 4, APRIL 2013 797 .”
Reconstruction Network for Video Captioning (2018, methodology): “For example, a concept hierarchy of actions was introduced to describe human activities in [17], while a semantic hierarchy was deﬁned in [10] to learn the semantic relationship between different sentence fragments.”