Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

By Ke Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron C. Courville, Ruslan Salakhutdinov, Richard S. Zemel, Yoshua Bengio. 2015

10,867 citations recorded by the source corpus.

Inspired by recent work in machine translation and object detection, we introduce an attention based model that automatically learns to describe the content of images. We describe how we can train this model in a deterministic manner using standard backpropagation techniques and stochastically by maximizing a variational lower bound. We also show through visualization how the model is able to automatically learn to fix its gaze on salient objects while generating the corresponding words in the output sequence. We validate the use of attention with state-of-the-art performance on three benchmark datasets: Flickr9k, Flickr30k and MS COCO.

Subfields: Computer Vision and Pattern Recognition, Natural Language Processing, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Ming-Hsuan Yang
  5. Xiaogang Wang
  6. Jitendra Malik
  7. Yi Yang
  8. Andrew Zisserman

In Natural Language Processing

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Graham Neubig
  4. Kyunghyun Cho
  5. Luke S. Zettlemoyer
  6. Yejin Choi
  7. Richard Socher
  8. Percy Liang

In Multimodal Learning

  1. Zhe Gan
  2. Devi Parikh
  3. Mohit Bansal
  4. Marcus Rohrbach
  5. Qi Wu
  6. Jianfeng Gao
  7. Trevor Darrell
  8. Dhruv Batra

Citations in the corpus