An Overview of Deep Semi-Supervised Learning

By Yassine Ouali, Céline Hudelot, Myriam Tami. arXiv.org, 2020

362 citations recorded by the source corpus.

Deep neural networks demonstrated their ability to provide remarkable performances on a wide range of supervised learning tasks (e.g., image classification) when trained on extensive collections of labeled data (e.g., ImageNet). However, creating such large datasets requires a considerable amount of resources, time, and effort. Such resources may not be available in many practical cases, limiting the adoption and the application of many deep learning methods. In a search for more data-efficient deep learning methods to overcome the need for large annotated datasets, there is a rising research interest in semi-supervised learning and its applications to deep neural networks to reduce the amount of labeled data required, by either developing novel methods or adopting existing semi-supervised learning frameworks for a deep learning setting. In this paper, we provide a comprehensive overview of deep semi-supervised learning, starting with an introduction to the field, followed by a summarization of the dominant semi-supervised approaches in deep learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/09a740aca5510c8cd5dd5a7474d6c1dcae08eff6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization (2023, methodology): “Each of these models is fine-tuned using the same data and hyperparameters as Alpaca [13].”
Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation (2022, background): “Then wrong predictions can be quickly amplified resulting in confident but erroneous pseudo labels during the training process [163].”
Data Feedback Loops: Model-driven Amplification of Dataset Biases (2022, result): “The semi-supervised learning setting (Ouali et al., 2020; Grandvalet & Bengio, 2004), also widely referred to as self-training, shares many similarities with the data feedback setting.”
A Brief Overview of Unsupervised Neural Speech Representation Learning (2022, background): “Selfsupervised learning is often characterized by automatically deriving the target from the input or other unlabeled examples (Ouali, Hudelot, and Tami 2020).”
A Survey on Deep Semi-Supervised Learning (2021, positive): “A recent review by Ouali et al. [31] gives a similar notion of DSSL as we do.”