Learning with Rejection

By Corinna Cortes, Giulia DeSalvo, Mehryar Mohri. International Conference on Algorithmic Learning Theory, 2016

341 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/86275934c44eec65ccead5a52a1c48c198de34ab

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Single-Model Uncertainties for Deep Learning (2018, background): “Abstention is a reasonable strategy to deal with anomalies [11], outliers [39], out-of-distribution examples [69], detect and defend against adversaries [75], or delegate high-risk predictions to humans [14, 27, 12].”
Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer (2017, methodology): “In this area, the standard method is to optimize the accuracy-rejection tradeoff: how much can a model improve its accuracy on the cases it does classify by PASS-ing some cases? This is usually learned by minimizing a classification objective Lreject with a penalty γreject for each rejection [10], w”
Predict Responsibly: Increasing Fairness by Learning To Defer (2017, methodology): “Rejection learning (Chow, 1957; Cortes et al., 2016) proposes a solution: allow models to reject (not make a prediction) when they are not confidently accurate.”
Generalized ODIN: Detecting Out-of-Distribution Image Without Learning From Out-of-Distribution Data (2020, background): “This problem has been studied for many years [12] and has been discussed in several views such as rejection [8, 5], anomaly detection [1], open set recognition [2], and uncertainty estimation [22, 23, 24].”
Does Your Model Know the Digit 6 Is Not a Cat? A Less Biased Evaluation of "Outlier" Detectors (2018, background): “r(x) = |h(x)| > τ [19] or chosen from a reject hypothesis set R [20, 21].”
The Relationship Between Agnostic Selective Classification Active Learning and the Disagreement Coefficient (2017, background): “Keywords: Active learning, selective prediction, disagreement coefficient, selective sampling, selective classification, reject option, pointwise-competitive, selective classification, statistical learning theory, PAC learning, sample complexity, agnostic case”
Consistent Estimators for Learning to Defer to an Expert (2020, methodology): “As we eluded to in the body of the paper, we can extend the losses introduced by [CDM16b] to our setting for binary labels.”
SelectiveNet: A Deep Neural Network with an Integrated Reject Option (2019, background): “The literature on reject option techniques is quite extensive and mostly focuses on hypothesis classes and learning algorithms such as SVMs, nearest neighbours and boosting (Fumera & Roli, 2002; Hellman, 1970; Cortes et al., 2016; El-Yaniv & Wiener, 2015).”
Is Out-of-Distribution Detection Learnable? (2022, background): “[55, 56, 57, 58, 59] study the learning theory and propose the PAC learning bounds for CwRO.”
Selective Brain Damage: Measuring the Disparate Impact of Model Pruning (2019, background): “PIE could be used to surface atypical examples for further human inspection [48], choose not to classify certain examples when the model is uncertain [2, 10, 9, 11], or to aid interpretability as a case based reasoning tool to explain model behavior [41, 23, 4, 33].”
Combating Label Noise in Deep Learning Using Abstention (2019, background): “Abstention classi-ﬁers have been proposed for shallow learners (Chow, 1970; Cortes et al., 2016; Fumera & Roli, 2002) and for multi-layer perceptrons (De Stefano et al., 2000).”
To Trust Or Not To Trust A Classifier (2018, background): “The problem of classification with a reject option or learning with abstention [16, 17, 18, 19, 20, 21, 22] is a highly related framework where the classifier is allowed to abstain from making a prediction at a certain cost.”
Learning under selective labels in the presence of expert consistency (2018, neutral): “Our work shares similarities with the literature on learning to defer (Cortes et al., 2016; Madras et al., 2017), which also combines human and algorithmic decision making.”
A Less Biased Evaluation of Out-of-distribution Sample Detectors (2018, background): “A reject function makes the abstention choice and it could be a threshold on magnitude of the prediction [1] or chosen from a reject-hypothesis set [4, 5].”