Mutual exclusivity loss for semi-supervised deep learning

By Mehdi S. M. Sajjadi, Mehran Javanmardi, Tolga Tasdizen. 2016

83 citations recorded by the source corpus.

In this paper we consider the problem of semi-supervised learning with deep Convolutional Neural Networks (ConvNets). Semi-supervised learning is motivated on the observation that unlabeled data is cheap and can be used to improve the accuracy of classifiers. In this paper we propose an unsupervised regularization term that explicitly forces the classifier's prediction for multiple classes to be mutually-exclusive and effectively guides the decision boundary to lie on the low density space between the manifolds corresponding to different classes of data. Our proposed approach is general and can be used with any backpropagation-based learning method. We show through different experiments that our method can improve the object recognition performance of ConvNets using unlabeled data.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence (2020, background): “This has led to a plethora of SSL methods that are designed for deep networks [28, 39, 21, 43, 3, 45, 2, 22, 38].”
Temporal Ensembling for Semi-Supervised Learning (2016, methodology): “The recently introduced transform/stability loss of Sajjadi et al. (2016b) is based on the same principle as our work, and the Π -model can be seen as a special case of it.”
Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning (2016, methodology): “A decoder is added to each layer of the noisy path.”
A Simple Semi-Supervised Learning Framework for Object Detection (2020, methodology): “Many advancements in SSL for classification rely on some forms of consistency regularization [27,38,25,44,54,35,4,58,3,59,49].”
Milking CowMask for Semi-Supervised Image Classification (2020, background): “Semi-supervised learning algorithms based on consistency regularization (Sajjadi et al., 2016a; Laine & Aila, 2017; Oliver et al., 2018) have proved to be simple while effective, yielding a number of state of the art results over the last few years.”
Label Propagation for Deep Semi-Supervised Learning (2019, background): “[15, 23, 34, 38], that add an unsupervised loss term (often called a regularizer) into the loss function.”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, background): “Although entropy minimization is originally proposed for logistic regression to impute the labels of samples classified with high confidence [92], it is later extended to train deep neural networks in SSL setting by minimizing the entropy of the class assignments either derived in the prediction spa”
Realistic Evaluation of Semi-Supervised Learning Algorithms (2018, methodology): “On its own, entropy minimization has not been shown to produce competitive results compared to the other methods described here (Sajjadi et al., 2016a).”
Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation (2022, background): “[104] introduce a mutual exclusivity loss for multi-class problems that explicitly forces the predictions to be mutually exclusive and encourages the decision boundary to lie on the low density space between the manifolds corresponding to different classes of data, which has a better performance in ”