Positive-Unlabeled Learning with Non-Negative Risk Estimator

By Ryuichi Kiryo, Gang Niu, Marthinus Christoffel du Plessis, Masashi Sugiyama. Neural Information Processing Systems, 2017

603 citations recorded by the source corpus.

From only positive (P) and unlabeled (U) data, a binary classifier could be trained with PU learning, in which the state of the art is unbiased PU learning. However, if its model is very flexible, empirical risks on training data will go negative, and we will suffer from serious overfitting. In this paper, we propose a non-negative risk estimator for PU learning: when getting minimized, it is more robust against overfitting, and thus we are able to use very flexible models (such as deep neural networks) given limited P data. Moreover, we analyze the bias, consistency, and mean-squared-error reduction of the proposed risk estimator, and bound the estimation error of the resulting empirical risk minimizer. Experiments demonstrate that our risk estimator fixes the overfitting problem of its unbiased counterparts.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/54db2ab1515661bbb6114d5fbd59b45b0644413e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding (2021, methodology): “In PU learning, a classiﬁer is learned using labeled data drawn from the positive class ( P ) and unlabeled data ( U ) that is a mixture of positive and negative samples with a positive class prior π (Du Plessis et al., 2014; Kiryo et al., 2017; Du Plessis & Sugiyama, 2014).”
On Positive-Unlabeled Classification in GAN (2020, background): “Identifying high-quality samples from generated samples under the guidance of real samples is very similar to Positive-Unlabeled classification problems [3, 12, 23], where only some positive samples were labeled, and the classifier tried to find positive samples from unlabeled samples consisting of ”
Off-Policy Evaluation via Off-Policy Classification (2019, methodology): “3) is to frame OPE as a positive-unlabeled (PU) classiﬁcation [16] problem, which provides for a way to derive OPE metrics that are both (a) fundamentally different from prior methods based on IS and model learning, and (b) perform well in practice on both simulated and real-world tasks.”
Noise-tolerant fair classification (2019, background): “, 2015), which are prominent noise models that have seen sustained study in recent years (Jain et al., 2017; Kiryo et al., 2017; van Rooyen & Williamson, 2018; Katz-Samuels et al., 2019; Charoenphakdee et al., 2019).”
Debiased Contrastive Learning (2020, methodology): “Our approach is related to unbiased PU learning, where the unlabeled data is used as negative examples, but down-weighted appropriately [26, 12, 11].”
Learning from positive and unlabeled data: a survey (2018, background): “2019) but it already existed before under the SCAR assumption (Steinberg and Scott Cardell 1992; Du Plessis et al. 2015a; Kiryo et al. 2017).”
On Symmetric Losses for Learning from Corrupted Labels (2019, background): “Other examples of weakly-supervised learning are learning from positive and unlabeled data (du Plessis et al., 2015; 2014; Kiryo et al., 2017), learning from pairwise similarity and unlabeled data (Bao et al.”
Positive-Unlabeled Reward Learning (2019, methodology): “if g is a deep neural network, leading to issues with overfitting [17].”
Combating Noisy Labels by Agreement: A Joint Training Method with Co-Regularization (2020, methodology): “These datasets are popularly used for the evaluation of learning with noisy labels in previous literatures [10, 18, 29].”
How does Disagreement Help Generalization against Label Corruption? (2019, methodology): “These datasets are popularly used for the evaluation of learning with noisy labels in the literature (Reed et al., 2015; Goldberger & Ben-Reuven, 2017; Kiryo et al., 2017).”
Is Out-of-Distribution Detection Learnable? (2022, background): “[29] mainly depends on the positive-unlabeled learning techniques [77, 78, 79].”