Classification with a Reject Option using a Hinge Loss

By Peter L. Bartlett, Marten H. Wegkamp. 2008

563 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Mechanics, Statistics and Probability, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Statistics and Probability

In Optimization & Training

Citations in the corpus

Natural Adversarial Examples (2019, methodology): “Before neural networks, using the reject option or a k+1st class was somewhat common [1], but with neural networks it requires auxiliary anomalous training data.”
Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness (2020, background): “When deep classifiers make predictions on input examples that are far from the support of the training set, their performance can be arbitrarily bad [4, 14].”
Generating Adversarial Examples with Adversarial Networks (2018, methodology): “To bound the magnitude of the perturbation, which is a common practice in prior work (Carlini & Wagner, 2017a; Liu et al., 2017; Bartlett & Wegkamp, 2008), we add a soft hinge loss on the L 2 norm as where c denotes a user-speciﬁed bound.”
Why ReLU Networks Yield High-Confidence Predictions Far Away From the Training Data and How to Mitigate the Problem (2018, background): “[3], will not help to detect these instances either.”
Deep Anomaly Detection with Outlier Exposure (2018, methodology): “An alternative OOD detection formulation is to give classifiers a “reject class” (Bartlett & Wegkamp, 2008).”
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, background): “…et al., 1990; Boult et al., 2019), a variety of practical approaches nevertheless proposed to simply rely on a hinge loss to reject during classiﬁcation (Bartlett and Wegkamp, 2008) or even to take the straightforward route and directly trust the softmax conﬁdence (Hendrycks and Gimpel, 2017).”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “…(Cordella et al., 1995; Geifman and El-Yaniv, 2017), to methods involving ensembles of models (Gal and Ghahramani, 2016; Lakshminarayanan et al., 2017; Geifman et al., 2018) or jointly learning to abstain and classify (Bartlett and Wegkamp, 2008; Geifman and El-Yaniv, 2019; Feng et al., 2019).”
To Trust Or Not To Trust A Classifier (2018, background): “The problem of classiﬁcation with a reject option or learning with abstention [3, 57, 9, 23, 8, 27, 10] is a highly related framework where the classiﬁer is allowed to abstain from making a prediction at a certain cost.”
Does Your Model Know the Digit 6 Is Not a Cat? A Less Biased Evaluation of "Outlier" Detectors (2018, background): “r(x) = |h(x)| > τ [19] or chosen from a reject hypothesis set R [20, 21].”
Asymmetric Loss For Multi-Label Classification (2020, methodology): “Linear hinge loss [1], for example, can also be seen as (symmetric) probability shifting of linear loss.”
Selective Classification Can Magnify Disparities Across Groups (2020, methodology): “Others incorporate abstention into model training (Bartlett & Wegkamp, 2008; Geifman & El-Yaniv, 2019; Feng et al., 2019) and learn to abstain on examples human experts are more likely to get correct (Raghu et al., 2019; Mozannar & Sontag, 2020; De et al., 2020).”
Scalable Joint Models for Reliable Uncertainty-Aware Event Prediction (2017, background): “Classiﬁcation with abstention has also been investigated before (see, e.g., [21, 22, 23]).”
On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation (2020, background): “It is also sometimes referred to as abstention, and the concept is not restricted to deep learning [BW08, CDM16, GECd18].”
Learning and the Unknown: Surveying Steps toward Open World Recognition (2019, background): “Related, but formally separate from open recognition, are the approaches that use classifiers with rejection (Chow 1970; Matan et al. 1990; Fumera and Roli 2002; Rong and Metaxas 2006; Bartlett and Wegkamp 2008; Grandvalet et al. 2008), novelty, anomaly or outlier detection (Hodge and Austin 2004; M”
A Less Biased Evaluation of Out-of-distribution Sample Detectors (2018, background): “A reject function makes the abstention choice and it could be a threshold on magnitude of the prediction [1] or chosen from a reject-hypothesis set [4, 5].”
CoDeL: A Human Co-detection and Labeling Framework (2013, background): “This confidence criterion essentially shares the same insight with the rejection option studied in GP classification [15, 3] and provides a very useful hint on finding positive data.”