A Comprehensive Survey of Deep Learning for Image Captioning

By Md. Zakir Hossain, Ferdous Sohel, Mohd Fairuz Shiratuddin, Hamid Laga. 2018

896 citations recorded by the source corpus.

Generating a description of an image is called image captioning. Image captioning requires recognizing the important objects, their attributes, and their relationships in an image. It also needs to generate syntactically and semantically correct sentences. Deep-learning-based techniques are capable of handling the complexities and challenges of image captioning. In this survey article, we aim to present a comprehensive review of existing deep-learning-based image captioning techniques. We discuss the foundation of the techniques to analyze their performances, strengths, and limitations. We also discuss the datasets and the evaluation metrics popularly used in deep-learning-based automatic image captioning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

Citations in the corpus

Multimodal Few-Shot Learning with Frozen Language Models (2021, methodology): “Following standard captioning systems [22, 13], we treat captioning as conditional generation of caption text y given an image x.”
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data (2020, background): “One approach to providing grounding is to train distributional models on corpora augmented with perceptual data, such as photos (Hossain et al., 2019) or other modalities (Kiela and Clark, 2015; Kiela et al., 2015).”
Evaluation of Text Generation: A Survey (2020, background): “Others survey specific NLG tasks or NLG models, such as image captioning (Kilickaya et al., 2017; Hossain et al., 2018; Li et al., 2019; Bai & An, 2018), machine translation (Dabre et al., 2020; Han & Wong, 2016; Wong & Kit, 2019), summarization (Deriu et al., 2009; Shi et al., 2018), question…”
Learning Dynamics and Heterogeneity of Spatial-Temporal Graph Data for Traffic Forecasting (2021, background): “It has been widely used and achieved great successes in various tasks such as natural language processing (NLP) [39], [40], image caption [41] and speech recognition [42].”
Foundation Models Defining a New Era in Vision: A Survey and Outlook (2025, neutral): “StandardCaptioning( Cap )loss[45] also aims to predict the next token given previous tokens and images.”
Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods (2019, methodology): “We present details for only eight tasks by excluding visual description generation (Aafaq et al., 2018; Hossain et al., 2019) and question answering (Wu et al.”
From Show to Tell: A Survey on Deep Learning-Based Image Captioning (2021, background): “Compared to previous surveys on image captioning [16], [17], [18], [21], [22], we provide a comprehensive and updated view on deep learning-based generative captioning models.”
Convolutional neural networks in medical image understanding: a survey (2021, background): “Researchers are also working on image captioning (textual representations of the image) [29].”
A Review of Deep Learning with Special Emphasis on Architectures, Applications and Recent Trends (2019, methodology): “Image Processing A Survey on Deep Learning in Medical Image Analysis [213] A Comprehensive Survey of Deep Learning for Image Captioning [242] Biological image analysis using deep learning-based methods: Literature review [243] Deep learning for remote sensing image classification: A survey [244] Dee”
Improved Few-Shot Visual Classification (2019, neutral): “Deep learning successes have led to major computer vision advances [11, 13, 37].”
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey (2024, methodology): “These enhanced models, namely visual instruct-following LLMs, can process both images and text to produce textual responses, which can be benchmarked on tasks such as image captioning [152], [204], [172], [74] and visual question answering (VQA) [176], [188], [4].”
Remaining Useful Lifetime Prediction via Deep Domain Adaptation (2019, background): “ for being able to extract spatial information from 2D and 3D high dimensional data yielding the best-known results in several related tasks such as image 4 segmentation, classication and captioning [31]. CNN’s can also handle 1D sequential data and extract high-level features by combining convoluti”
Empowering Things With Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things (2020, background): “video [179] while text-to-image generation aims to generate a realistic image that matches the given text description [180].”