A Review on Deep Learning Techniques Applied to Semantic Segmentation

By Alberto Garcia-Garcia, Sergio Orts, Sergiu Oprea, Victor Villena-Martinez, Jose García Rodríguez. 2017

1,387 citations recorded by the source corpus.

Image semantic segmentation is more and more being of interest for computer vision and machine learning researchers. Many applications on the rise need accurate and efficient segmentation mechanisms: autonomous driving, indoor navigation, and even virtual or augmented reality systems to name a few. This demand coincides with the rise of deep learning approaches in almost every field or application target related to computer vision, including semantic segmentation or scene understanding. This paper provides a review on deep learning methods for semantic segmentation applied to various application areas. Firstly, we describe the terminology of this field as well as mandatory background concepts. Next, the main datasets and challenges are exposed to help researchers decide which are the ones that best suit their needs and their targets. Then, existing methods are reviewed, highlighting their contributions and their significance in the field. Finally, quantitative results are given for the described methods and the datasets in which they were evaluated, following up with a discussion of the results. At last, we point out a set of promising future works and draw our own conclusions about the state of the art of semantic segmentation using deep learning techniques.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Knowledge Distillation for Incremental Learning in Semantic Segmentation (2019, methodology): “Recent reviews on this topic are [18, 19, 20].”
Affinity Derivation and Graph Merge for Instance Segmentation (2018, background): “Rapid progress has been made in the areas of object detection [14, 26, 43], semantic segmentation [17], and even instance segmentation [19, 21].”
Local Temperature Scaling for Probability Calibration (2020, background): “However, due to overfitting, CNNs for semantic segmentation tend to be overconfident about predicted labels [17, 20, 29, 41].”
Understanding Deep Learning Techniques for Image Segmentation (2019, methodology): “The most classic version is semantic segmentation [66].”
Kimera: From SLAM to spatial perception with 3D dynamic scene graphs (2021, background): “, 2017)) and deep-learning-based semantic segmentation (e.g., Badrinarayanan et al., 2017; Garcia-Garcia et al., 2017; He et al., 2017; Hu et al., 2017; Krizhevsky et al., 2012; Redmon and Farhadi, 2017; Ren et al., 2015), research in these two fields has traditionally proceeded in isolation, and th”
A review of deep learning used in the hyperspectral image analysis for agriculture (2021, background): “…deep learning models have recently been successfully adopted in various areas such as computer vision (Dan et al. 2011; Athanasios et al. 2018; Voulodimos et al. 2018), speech recognition (Hinton et al. 2012), and other similarly complex data processing (Garcia-Garcia et al. 2017; Guo et al. 2016).”
A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification (2018, methodology): “Data augmentation is a well-accepted process that has been recently used to increase the size of the dataset, speed up the convergence, and avoid overfitting problems [10,19,24,33,54,55].”
Continual Learning of Semantic Segmentation using Complementary 2D-3D Data Representations (2021, background): “In semantic segmentation, such domain gaps commonly cause misclassification of unknown categories and wrong labeling of small semantic details in favor of the dominant neighboring semantic class [1].”
Deep semantic segmentation of natural and medical images: a review (2019, background): “Compared to the other review papers on deep learning-based semantic image segmentation for natural and medical images [35, 41, 47, 52, 71, 140, 179], we make the following contributions: • We provide comprehensive coverage of research contributions in the ﬁeld of natural and medical image semantic…”
Pathology Image Analysis Using Segmentation Deep Learning Algorithms. (2019, methodology): “34 Other neural network e based semantic segmentation methods have been discussed by Garcia-Garcia et al. 35 The goal of instance segmentation is to segment different objects regardless of whether the objects belong to the same category or not.”
Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping (2019, background): “, [4]–[10]), research in these two fields has traditionally proceeded in isolation.”
RTSeg: Real-Time Semantic Segmentation Comparative Study (2018, background): “The survey on semantic segmentation [13] presented a comparative study between different segmentation architectures including ENet [12].”
Multi-Task Learning for Segmentation of Building Footprints with Deep Neural Networks (2017, background): “A detailed survey of deep neural network based architectures for semantic segmentation can be found in [6].”
PseudoClick: Interactive Image Segmentation with Click Imitation (2022, background): “g, semantic object/instance segmentation [1,2], video understanding [3,4], autonomous driving [5,6,7], and medical image analysis [8,9].”
Semantic Segmentation of Underwater Imagery: Dataset and Benchmark (2020, methodology): “Semantic segmentation is a well-studied problem in the domains of robot vision and deep learning [1], [2], [3] for its usefulness in estimating scene geometry, inferring interactions and spatial relationships among objects, salient object identification, and more.”
Automatic crack classification and segmentation on masonry surfaces using convolutional neural networks and transfer learning (2021, methodology): “A review on DL methods for semantic segmentation applied to various application areas was presented in [21].”
Computing Systems for Autonomous Driving: State of the Art and Challenges (2020, background): “It provides several benchmarks, such as stereo, flow, scene, optical flow, depth, odometry, object tracking [55], road, and semantics [56].”