Book2Movie: Aligning video scenes with book chapters

By Makarand Tapaswi, Martin Bäuml, Rainer Stiefelhagen. 2015

93 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

In Signal Processing

Citations in the corpus

Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books (2015, background): “Parallel to our work, [37] aims to align scenes in movies to chapters in the book.”
MovieQA: Understanding Stories in Movies through Question-Answering (2015, background): “Just recently, [38, 49] aligned movies to books with the aim to ground temporal visual data with verbose and detailed descriptions available in books.”
Grounding of Textual Phrases in Images by Reconstruction (2015, background): “associating a paragraph of text to a scene in a movie [30, 41], to fine, e.”
Weakly-Supervised Alignment of Video with Text (2015, background): “The problem of aligning a large text corpus with video is addressed in [45].”
Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods (2019, unknown): “Initially, aligning books to movies (Tapaswi et al., 2015; Zhu et al., 2015) is used to generate story like explanations.”
Action Modifiers: Learning From Adverbs in Instructional Videos (2019, methodology): “Movies accompanied by subtitles and scripts have been used for learning from video [12, 13, 25, 47].”
To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regression (2018, methodology): “computed an alignment between book chapters and movie scenes using matching dialogs and character identities as cues with a graph based algorithm [Tapaswi et al., 2015].”
Speech2Action: Cross-Modal Supervision for Action Recognition (2020, background): “A similar argument can be applied to works that align books to movies [40, 52].”
COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learning (2020, background): “Several works tried to utilize the temporal structure of video and text for the alignment task [77, 70, 78, 4, 79].”
Condensed Movies: Story Based Retrieval with Contextual Embeddings (2020, methodology): “Hence inspired by [49], we also propose a second method (“track-frequency normalised”), where xi is the number of face tracks for identity i.”
Textually Customized Video Summaries (2017, methodology): “Our textually customized video summarization approach is motivated by recent developments on joint embedding of image and text [5, 13, 22, 27, 33, 36].”
Neural Task Graphs: Generalizing to Unseen Tasks From a Single Video Demonstration (2018, unknown): “This includes instructional video [2, 18, 34], movie script [37, 45], and caption annotation [14, 25].”