Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures

By Raffaella Bernardi, Ruken Cakici, Desmond Elliott, Aykut Erdem, Erkut Erdem, Nazli Ikizler-Cinbis, Frank Keller, Adrian Muscat, Barbara Plank. 2016

393 citations recorded by the source corpus.

Automatic description generation from natural images is a challenging problem that has recently received a large amount of interest from the computer vision and natural language processing communities. In this survey, we classify the existing approaches based on how they conceptualize this problem, viz., models that cast description as either generation problem or as a retrieval problem over a visual or multimodal representational space. We provide a detailed review of existing models, highlighting their advantages and disadvantages. Moreover, we give an overview of the benchmark image datasets and the evaluation measures that have been developed to assess the quality of machine-generated image descriptions. Finally we extrapolate future directions in the area of automatic image description generation.

Subfields: Computer Vision and Pattern Recognition, Multimodal Learning, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Ming-Hsuan Yang
  5. Xiaogang Wang
  6. Yi Yang
  7. Andrew Zisserman
  8. Jitendra Malik

In Multimodal Learning

  1. Desmond Elliott
  2. Rita Cucchiara
  3. Lorenzo Baraldi
  4. Marcella Cornia
  5. Loïc Barrault
  6. Lucia Specia
  7. Dietrich Klakow
  8. Ozan Caglayan

In Natural Language Generation

  1. Rita Cucchiara
  2. Lorenzo Baraldi
  3. Marcella Cornia
  4. Giuseppe Fiameni
  5. Matteo Stefanini
  6. Marimuthu Kalimuthu
  7. Dietrich Klakow
  8. Aditya Mogadala

Citations in the corpus