Convolutional Image Captioning

By Jyoti Aneja, Aditya Deshpande, Alexander Gerhard Schwing. 2017

401 citations recorded by the source corpus.

Image captioning is an important task, applicable to virtual assistants, editing tools, image indexing, and support of the disabled. In recent years significant progress has been made in image captioning, using Recurrent Neural Networks powered by long-short-term-memory (LSTM) units. Despite mitigating the vanishing gradient problem, and despite their compelling ability to memorize dependencies, LSTM units are complex and inherently sequential across time. To address this issue, recent work has shown benefits of convolutional networks for machine translation and conditional image generation [9, 34, 35]. Inspired by their success, in this paper, we develop a convolutional image captioning technique. We demonstrate its efficacy on the challenging MSCOCO dataset and demonstrate performance on par with the LSTM baseline [16], while having a faster training time per number of parameters. We also perform a detailed analysis, providing compelling reasons in favor of convolutional language generation approaches.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Percy Liang
  2. Mohit Bansal
  3. Jianfeng Gao
  4. Ruslan Salakhutdinov
  5. Yoshua Bengio
  6. Quoc V. Le
  7. Dacheng Tao
  8. Caiming Xiong

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Yi Yang
  4. Xiaogang Wang
  5. Trevor Darrell
  6. Ming-Hsuan Yang
  7. Jitendra Malik
  8. Andrew Zisserman

In Multimodal Learning

  1. Rita Cucchiara
  2. Lorenzo Baraldi
  3. Marcella Cornia
  4. Percy Liang
  5. Matteo Stefanini
  6. Giuseppe Fiameni
  7. Louis-philippe Morency
  8. Jiebo Luo

In Natural Language Generation

  1. Rita Cucchiara
  2. Lorenzo Baraldi
  3. Marcella Cornia
  4. Matteo Stefanini
  5. Giuseppe Fiameni
  6. Idan Schwartz
  7. Mohammed Bany Muhammad
  8. Mohammed Yeasin

Citations in the corpus