Contrastive Pessimistic Likelihood Estimation for Semi-Supervised Classification

By Marco Loog. 2015

70 citations recorded by the source corpus.

Improvement guarantees for semi-supervised classifiers can currently only be given under restrictive conditions on the data. We propose a general way to perform semi-supervised parameter estimation for likelihood-based classifiers for which, on the full training set, the estimates are never worse than the supervised solution in terms of the log-likelihood. We argue, moreover, that we may expect these solutions to really improve upon the supervised classifier in particular cases. In a worked-out example for LDA, we take it one step further and essentially prove that its semi-supervised version is strictly better than its supervised counterpart. The two new concepts that form the core of our estimation principle are contrast and pessimism. The former refers to the fact that our objective function takes the supervised estimates into account, enabling the semi-supervised solution to explicitly control the potential improvements over this estimate. The latter refers to the fact that our estimates are conservative and therefore resilient to whatever form the true labeling of the unlabeled data takes on. Experiments demonstrate the improvements in terms of both the log-likelihood and the classification error rate on independent test sets.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Machine learning approaches in medical image analysis: From detection to diagnosis (2016, background): “Transfer learning techniques that could guarantee that the result of the transfer technique is never worse than the supervised solution, such as recently proposed for semi-supervised learning (Loog, 2016), are therefore of great interest.”
Improvability Through Semi-Supervised Learning: A Survey of Theoretical Results (2019, background): “We will speciﬁcally look at work from Li and Zhou (2011) and Loog (2016).”
Robustness to Adversarial Perturbations in Learning from Incomplete Data (2019, methodology): “We show that a special case of our method reduces to an adversarial extension of [32].”
Projected estimators for robust semi-supervised classification (2016, methodology): “From this perspective the projected estimator is similar to Maximum Contrastive Pessimistic Likelihood Estimation proposed by Loog (2016) who consider using log likelihood as the loss function.”
Dash: Semi-Supervised Learning with Dynamic Thresholding (2021, background): “An non-exhaustive list of those studies include (Cozman et al., 2003; Singh et al., 2009; Li & Zhou, 2011a; Balsubramani & Freund, 2015; Loog, 2015; Li et al., 2017; Krijthe & Loog, 2017; Li et al., 2021; Mey & Loog, 2019; Guo et al., 2020).”