From Show to Tell: A Survey on Deep Learning-Based Image Captioning

By Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Silvia Cascianelli, Giuseppe Fiameni, Rita Cucchiara. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021

395 citations recorded by the source corpus.

Connecting Vision and Language plays an essential role in Generative Intelligence. For this reason, large research efforts have been devoted to image captioning, i.e. describing images with syntactically and semantically meaningful sentences. Starting from 2015 the task has generally been addressed with pipelines composed of a visual encoder and a language model for text generation. During these years, both components have evolved considerably through the exploitation of object regions, attributes, the introduction of multi-modal connections, fully-attentive approaches, and BERT-like early-fusion strategies. However, regardless of the impressive results, research in image captioning has not reached a conclusive answer yet. This work aims at providing a comprehensive overview of image captioning approaches, from visual encoding and text generation to training strategies, datasets, and evaluation metrics. In this respect, we quantitatively compare many relevant state-of-the-art approaches to identify the most impactful technical innovations in architectures and training strategies. Moreover, many variants of the problem and its open challenges are discussed. The final goal of this work is to serve as a tool for understanding the existing literature and highlighting the future directions for a research area where Computer Vision and Natural Language Processing can find an optimal synergy.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/1c83f3f9789df43bf937ae2618721e2da83dcc06

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Lichao Sun
2. Philip S. Yu
3. Zhiling Yan
4. Siyu Li
5. Yihan Cao
6. Yutong Dai
7. Charu C. Aggarwal
8. Bing Liu

In Computer Vision and Pattern Recognition

1. Philip S. Yu
2. Zhiling Yan
3. Siyu Li
4. Yihan Cao
5. Yutong Dai
6. Lichao Sun
7. Charu C. Aggarwal
8. Pavel Berkhin

Citations in the corpus

• A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT (2023, result): “However, the paradigm of GAI models containing the above two steps is not entirely novel, as demonstrated by previous studies [6, 7].”