Guiding the Long-Short Term Memory Model for Image Caption Generation

By Xu Jia, Efstratios Gavves, Basura Fernando, Tinne Tuytelaars. 2015

379 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning (2016, methodology): “Soft-Attention [29] 67.0 44.8 29.9 19.5 18.9 66.7 43.4 28.8 19.1 18.5 70.7 49.2 34.4 24.3 23.9 Hard-Attention [29] 67.0 45.7 31.4 21.3 20.3 66.9 43.9 29.6 19.9 18.5 71.8 50.4 35.7 25.0 23.0 emb-gLSTM [11] 64.7 45.9 31.8 21.2 20.6 64.6 44.6 30.5 20.6 17.9 67.0 49.1 35.8 26.4 22.7 ATT [32]y – – – – – ”
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression (2017, background): “, object detection [7], semantic segmentation [24], image captioning [16] and many others.”
Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures (2016, background): “Gong et al. (2014) propose an approach based on weak supervision that transfers knowledge from millions of weakly annotated images to improve the accuracy of description retrieval.”
VideoLSTM convolves, attends and flows for action recognition (2016, neutral): “LSTM networks (Hochreiter and Schmidhuber, 1997) model sequential memories both in the long and in the short term, which makes them relevant for various sequential tasks (Donahue et al., 2015; Jia et al., 2015; Karpathy and Fei-Fei, 2015; Wu et al., 2015; Yue-Hei Ng et al., 2015).”
A Comprehensive Survey of Deep Learning for Image Captioning (2018, background): “[21, 59, 61, 144, 150, 152] have performed experiments using the dataset.”
Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning (2017, background): “Other lines of research generate image captions using semantic information or via a compositional approach (Fang et al., 2015; Gan et al., 2017; Tran et al., 2016; Jia et al., 2015; Wu et al., 2016; You et al., 2016).”
From Show to Tell: A Survey on Deep Learning-Based Image Captioning (2021, methodology): “Global CNN features were then employed in a large variety of image captioning models [30], [31], [32], [33], [34], [35], [36], [37].”
Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event Detection (2017, background): “…but when doing long term path planning, even though the LSTM is capable of handling long term relationships, the prediction process may go almost “blindly” towards the end of the sequence (Jia et al. (2015)) as we are neglecting vital information about pedestrian’s behaviour under varying contexts.”
Video Captioning With Attention-Based LSTM and Semantic Consistency (2017, background): “Thanks to the rapid development of deep Convolutional Neural Network (CNN) [10], [11], recent works have made signiﬁcant progress for image description generation [8], [12], [13], [14], [15].”
A Comprehensive Survey on Cross-modal Retrieval (2016, background): “Another topic for modeling multimodal data is image/video description [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], which is not discussed here because it goes beyond the scope of cross-modal retrieval research.”
Online Action Detection (2016, neutral): “sider Fisher vectors [17] with improved trajectories [1], a deep ConvNet operating on a single frame basis [18] and an LSTM network, recently popular for sequential modelling such as image captioning [19] and action recognition [20], to encode the actions temporally. As it turns out, detecting actio”
StyleNet: Generating Attractive Visual Captions with Styles (2017, background): “Recent successes of using neural networks in image classification [26, 43, 40, 17] , object detection [16, 15, 39] and attribute learning [12] motivates strong interests in using neural networks for image captioning [50, 32, 22, 5, 21, 10, 9, 52, 54, 55, 2, 46].”
Deep learning for visual understanding (2017, neutral): “[307] added extra semantic information to each unit of the LSTM block.”
Extending Long Short-Term Memory for Multi-View Structured Learning (2016, methodology): “Extensions have been proposed using conventional LSTM for early fusion of language and images during decoding [7,26].”
A Joint Speaker-Listener-Reinforcer Model for Referring Expressions (2016, methodology): “One paper related to our work is gLSTM [11] which uses CCA semantics to guide the caption generation.”
Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods (2019, background): “Recently, pointing LSTM is designed to point the novel objects (Li et al., 2019) by balancing generation and copying words.”
HP-GAN: Probabilistic 3D Human Motion Prediction via GAN (2017, methodology): “They have been successfully used in machine translation [36], caption generation from images [17, 6], video classification and action recognition[2, 26, 6], action detection[42], video description[27, 19, 41, 6], sequence prediction[9, 35] and others.”