Video Captioning With Attention-Based LSTM and Semantic Consistency

By Lianli Gao, Zhao Guo, Hanwang Zhang, Xing Xu, Heng Tao Shen. IEEE transactions on multimedia, 2017

589 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/51b2c1e750b1d3b893072829d012f2352d6bd373

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Joint Syntax Representation Learning and Visual Cue Translation for Video Captioning (2019, background): “[ 48, 41, 13] introduce the attention mechanism for video captioning, where [48, 41] select salient spatial-temporal features to generate sentences and [13] uses hierarchical attention to capture the temporal dynamics for captioning.”
RiFeGAN: Rich Feature Generation for Text-to-Image Synthesis From Prior Knowledge (2020, methodology): “Attention-based models have been applied successfully in many areas, like the dual task, captioning [7, 15].”
Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning (2020, methodology): “Baseline Model: We compare our method with strong video captioning baseline models like, S2VT (Venugopalan et al., 2015), “Attention-EncDec” (Gao et al., 2017) – LSTM based models which reach competitive performing on MSR-VTT dataset. and “Dense Captioning” (Zhou et al., 2018), which is a transforme”
Deep Multimodal Representation Learning: A Survey (2019, methodology): “Recently, this method has demonstrated its unique power in improving performance in many applications such as visual classification [152]–[154], neural machine translation [155], [156], speech recognition [92], image captioning [13], [91], video description [42], [90], visual question-answering [24]”
The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers (2018, background): “In recent years, deep learning (DL) has witnessed great success in a wide spectrum of “smart” applications, such as video captioning [46], video event recognition [47], human action regnition [48], visual question answering [49], and question answering with knowledge base [50].”
Reconstruction Network for Video Captioning (2018, methodology): “aLSTMs [9] 50.”
Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods (2019, background): “…with external corpora (Venugopalan et al., 2016) and also attention with various techniques such as soft-attention (Yao et al., 2015), multimodal fusion (Hori et al., 2017), temporal attention (Song et al., 2017), semantic consistency (Gao et al., 2017) and residual connections (Li et al., 2018).”
MirrorGAN: Learning Text-To-Image Generation by Redescription (2019, background): “dresses visual similarities. Attention models have been extensively exploited in computer vision and natural language processing, for instance in object detection [21,6,18,40], image/video captioning [33,9,31], visual question answering [2,32,35, 22], and neural machine translation [19,8]. Attention”
Perceptual Pyramid Adversarial Networks for Text-to-Image Synthesis (2019, background): “Recently, we have witnessed a breakthrough in the application of deep learning to generate textual descriptions conditioned on images/videos (Song et al. 2017; Gao et al. 2017).”
Audio Visual Scene-Aware Dialog (2019, background): “It also expresses the spatio-temporal relationship between the dynamics in the scene [12,29,38].”
Binary Generative Adversarial Networks for Image Retrieval (2017, background): “More recently, deep learning has been introduced in the development of hashing algorithms (Xia et al. 2014; Lin et al. 2016; Do, Doan, and Cheung 2016; Gu, Ma, and Yang 2016; Wang et al. 2017a; Gao et al. 2017; Song et al. 2018), leading to a new generation of deep hashing algorithms.”
Beyond RNNs: Positional Self-Attention with Co-Attention for Video Question Answering (2019, methodology): “…2018a; Wang et al. 2018) are primarily based on RNNs, especially Long Short-Term Memory (LSTM) (Hochreiter and Schmidhuber 1997; Song et al. 2017; Gao et al. 2017) and Gated Recurrent Unit (GRU), which have good performance on modeling sequential data and potential to solve the gradient…”
Learning Heuristics for the TSP by Policy Gradient (2018, background): “[14] captioning, machine translation [15], speech recognition [16] and question answering [17].”
Object-Aware Aggregation With Bidirectional Temporal Graph for Video Captioning (2019, unknown): “9 aLSTMs [35] 50.”
COCO-CN for Cross-Lingual Image Tagging, Captioning, and Retrieval (2018, background): “[2] propose a novel attentionbased LSTM model with semantic consistency to explore the correlation between multi-modal representations for generating sentences with rich semantic content.”
A General Survey on Attention Mechanisms in Deep Learning (2022, background): “where attention can be applied intuitively [26], [27].”