A survey on semi-supervised learning

By Jesper E. van Engelen, Holger H. Hoos. 2019

2,517 citations recorded by the source corpus.

Semi-supervised learning is the branch of machine learning concerned with using labelled as well as unlabelled data to perform certain learning tasks. Conceptually situated between supervised and unsupervised learning, it permits harnessing the large amounts of unlabelled data available in many use cases in combination with typically smaller sets of labelled data. In recent years, research in this area has followed the general trends observed in machine learning, with much attention directed at neural network-based models and generative learning. The literature on the topic has also expanded in volume and scope, now encompassing a broad spectrum of theory, algorithms and applications. However, no recent surveys exist to collect and organize this knowledge, impeding the ability of researchers and engineers alike to utilize it. Filling this void, we present an up-to-date overview of semi-supervised learning methods, covering earlier work as well as more recent advances. We focus primarily on semi-supervised classification, where the large majority of semi-supervised learning research takes place. Our survey aims to provide researchers and practitioners new to the field as well as more advanced readers with a solid understanding of the main approaches and algorithms developed over the past two decades, with an emphasis on the most prominent and currently relevant work. Furthermore, we propose a new taxonomy of semi-supervised classification algorithms, which sheds light on the different conceptual and methodological approaches for incorporating unlabelled data into the training process. Lastly, we show how the fundamental assumptions underlying most semi-supervised learning algorithms are closely connected to each other, and how they relate to the well-known semi-supervised clustering assumption.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Novel Class Discovery without Forgetting (2022, background): “In semi-supervised learning approaches [10,51], the labeled and unlabeled data comes from the same set of classes.”
AI-big data analytics for building automation and management systems: a survey, actual challenges and future perspectives (2022, background): “It falls between unsupervised learning and supervised learning, which is also considered as a special instance of weak supervision (Van Engelen and Hoos 2020).”
Dash: Semi-Supervised Learning with Dynamic Thresholding (2021, methodology): “Numerous empirical evidences show that unlabeled data in SSL can help to improve the learning performance [Berthelot et al., 2019b,a, Sohn et al., 2020], however, a series of theoretical studies [Ben-David et al., 2008, Singh et al., 2009, Li and Zhou, 2011b, Balcan and Blum, 2005] have demonstrated”
Hyperspectral Image Classification—Traditional to Deep Models: A Survey for Future Prospects (2021, background): “A recent comprehensive survey on these SSL techniques can be found in [214].”
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, positive): “To adapt a pre-trained model to an unlabeled target domain, a majority of SFDA methods take inspiration from the semisupervised learning (SSL) field [146] and employ various”
Artificial Intelligence for the Metaverse: A Survey (2022, background): “Some wellknown algorithms for semi-supervised learning are graphbased model, generative model, boosting, and self-training [8].”
Machine Learning for Reliability Engineering and Safety Applications: Review of Current Status and Future Opportunities (2020, methodology): “Using that information, labels can be propagated transitively form a labeled data point to an unlabeled data point” [54].”
Self-Training: A Survey (2022, neutral): “While various surveys have explored semi-supervised learning in recent years [83, 92], none have specifically emphasized self-training, which has emerged as the predominant approach in the field, widely applied across various applications.”
Self-Supervised Representation Learning: Introduction, advances, and challenges (2021, neutral): “This setting can also be addressed by semi-supervised meth-ods [20], which should then be evaluated as competitors against SSRL. ■ If the target domain of interest is similar enough to large source data sets (e.g., everyday objects versus ImageNet), then one can leverage self-supervised pretraining…”
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review (2023, methodology): “Semisupervised techniques include expectation maximization with generative mixture models, transductive SVMs, graph-based methods, and so on [39], [40].”
Semi-Automated Data Labeling (2020, methodology): “To implement a machine labeler we use a semi-supervised learning (SSL) Van Engelen and Hoos (2020) algorithm.”
A Survey on Deep Semi-Supervised Learning (2021, result): “Although [30] tries to present a whole picture of SSL, the taxonomy is quite different from ours.”
Empowering Things With Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things (2020, background): “SSL refers to learning from both labeled and unlabeled data [195].”
CoMatch: Semi-supervised Learning with Contrastive Graph Regularization (2020, background): “More comprehensive reviews can be found in [44, 36].”
AbdomenCT-1K: Is Abdominal Organ Segmentation a Solved Problem? (2020, background): “It is a promising and active research direction in machine learning [61] as well as medical image analysis [10].”