A Survey on Deep Semi-Supervised Learning

By Xiangli Yang, Zixing Song, Irwin King, Zenglin Xu. 2021

910 citations recorded by the source corpus.

Deep semi-supervised learning is a fast-growing field with a range of practical applications. This paper provides a comprehensive survey on both fundamentals and recent advances in deep semi-supervised learning methods from perspectives of model design and unsupervised loss functions. We first present a taxonomy for deep semi-supervised learning that categorizes existing methods, including deep generative methods, consistency regularization methods, graph-based methods, pseudo-labeling methods, and hybrid methods. Then we provide a comprehensive review of 60 representative methods and offer a detailed comparison of these methods in terms of the type of losses, architecture differences, and test performance results. In addition to the progress in the past few years, we further discuss some shortcomings of existing methods and provide some tentative heuristic solutions for solving these open problems.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards Label-Efficient Incremental Learning: A Survey (2023, background): “Semi-supervision reduces the label need by leveraging unlabeled data [Yang et al., 2022].”
Machine Learning in Aerodynamic Shape Optimization (2022, background): “Semisupervised learning methods work with a small amount of labeled data and a large amount of unlabeled data [216, 217].”
Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation (2022, background): “The three basic assumptions for semi-supervised learning include [204, 205]: The Cluster Assumption.”
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, neutral): “Except for the pseudo-labeling paradigm, nearly all semi-supervised learning algorithms rely on the cluster assumption [146], which asserts that the decision boundary should not cross high-density regions, but instead lie in low-density regions.”
Federated learning for medical image analysis: A survey (2023, background): “Semi-supervised learning (Yang et al., 2022; Van Engelen and Hoos, 2020) is a popular solution for such scenarios.”
Self-Training: A Survey (2022, neutral): “While various surveys have explored semi-supervised learning in recent years [83, 92], none have specifically emphasized self-training, which has emerged as the predominant approach in the field, widely applied across various applications.”