The Extreme Value Machine

By Ethan M. Rudd, Lalit P. Jain, Walter J. Scheirer, Terrance E. Boult. 2015

346 citations recorded by the source corpus.

It is often desirable to be able to recognize when inputs to a recognition function learned in a supervised manner correspond to classes unseen at training time. With this ability, new class labels could be assigned to these inputs by a human operator, allowing them to be incorporated into the recognition function—ideally under an efficient incremental update mechanism. While good algorithms that assume inputs from a fixed set of classes exist, e.g. , artificial neural networks and kernel machines, it is not immediately obvious how to extend them to perform incremental learning in the presence of unknown query classes. Existing algorithms take little to no distributional information into account when learning recognition functions and lack a strong theoretical foundation. We address this gap by formulating a novel, theoretically sound classifier—the Extreme Value Machine (EVM). The EVM has a well-grounded interpretation derived from statistical Extreme Value Theory (EVT), and is the first classifier to be able to perform nonlinear kernel-free variable bandwidth incremental learning. Compared to other classifiers in the same deep network derived feature space, the EVM is accurate and efficient on an established benchmark partition of the ImageNet dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Wireless Network Intelligence at the Edge (2018, methodology): “The solution is a kernel-free recognition technique known as extreme value machine [151].”
FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learning (2022, neutral): “We present the main components of FeTrIL in the next subsections.”
Recent Advances in Open Set Recognition: A Survey (2018, methodology): “Under different openness O∗, Table 3 reports the comparisons among these methods, where 1-vs-Set [21], W-SVM (WOSVM7) [22], PI -SVM [23], SROSR [61], OSNN [73], EVM [74] from Traditional ML-based category and CD-OSR [63] from the Non-Instance Generation-based category.”
Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancy (2019, result): “There are some other studies on open-set classification [1, 2, 6, 21, 23, 24, 30], which involve tasks very similar to OOD detection.”
Distinction Maximization Loss: Fast, Scalable, Turnkey, and Native Neural Networks Out-of-Distribution Detection simply by Replacing the SoftMax Loss (2019, background): “essity from a practical point of view. The mentioned problem has been studied under many similar point of views and nomenclatures such as Open Set Recognition [2], [3] and Open World Recognition [4], [5]. Recently, [6] deﬁned as out-of-distribution (OOD) detection the task of evaluating whether a sa”
Learning and the Unknown: Surveying Steps toward Open World Recognition (2019, methodology): “Our more recent EVM model (Rudd et al. 2018), also provides a nonlinear radial basis function approach based on EVT, but it provides a more flexible and more powerful representation model for OpenMax than the single class mean used in the original OpenMax paper.”
Learning Bounds for Open-Set Learning (2021, methodology): “To address this challenge, different strategies have been proposed such as open-space risk (Scheirer et al., 2013) and extreme value theory (Jain et al., 2014; Rudd et al., 2018).”
Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning (2020, background): “…which is often used to simulate extreme events in the tails of one-dimensional probability distributions by assessing the probability of events that are more extreme (Broadwater and Chellappa 2010; Rudd et al. 2018; Al-Behadili, Grumpe, and W ¨ ohler 2015; Clifton et al. 2008; Shi et al. 2008).”
Adversarial Reciprocal Points Learning for Open Set Recognition (2021, background): “proposed extreme value machines [14] which modeled class-inclusion probabilities with an extreme value theory (EVT) based density function.”
Dropout Sampling for Robust Object Detection in Open-Set Conditions (2017, methodology): “own object as a known class with high conﬁdence [14]. Current attempts at improving open-set performance of machine learning systems have focused on formally accounting for unknown unknowns [4], [5], [15] by identifying and rejecting classes not encountered during training based on an estimate of th”
Extreme Value Theory for Open Set Classification - GPD and GEV Classifiers (2018, methodology): “In this section we compare the EVM introduced in [8] with our GPDC and the GEVC on simulated and real data.”
Memory Based Online Learning of Deep Representations from Video Streams (2017, background): “For-malization for open space risk is considered as the relative measure of open space compared to the overall measure space [15, 16, 17, 18].”
Class Boundary Exemplar Selection Based Incremental Learning for Automatic Target Recognition (2020, methodology): “Classiﬁers are used for experimental evaluation: nearest neighbor classiﬁer (1NN), support vector machine with radial basis function kernel function (SVM_RBF), linear support vector machine (SVM_Lin), and extreme value machine (EVM) [21].”
Principled Uncertainty Estimation for Deep Neural Networks (2018, methodology): “Rudd et al. (Rudd et al. 2018) and Bendale and Boult (Bendale and Boult 2016) applied post-hoc open set probability estimators in a deep feature space, but these models are not jointly optimized with the network.”
Classification-Reconstruction Learning for Open-Set Recognition (2018, methodology): “Extreme value machines [34] model class-inclusion probabilities using an extreme-value-theory-based density function.”
A Survey of Stealth Malware Attacks, Mitigation Measures, and Steps Toward Autonomous Open World Solutions (2016, methodology): “Both [144] and [143] apply EVT modeling to open set recognition scenarios based posterior fitting of point distances to classifier decision boundaries, while [160] incorporated EVT calibration into a generative model, which performs loose density estimation as a mixture of EVT distributions.”
Self-Supervised Features Improve Open-World Learning (2021, methodology): “However, that paper and subsequent papers on the topic [54], have”
Generalized Out-of-Distribution Detection: A Survey (2021, neutral): “The following research applies extreme value theory to OSL (Jain et al., 2014; Rudd et al., 2017).”
Continual egocentric object recognition (2019, background): “This is consistent with the recent trend in few-shot [11, 32] and open-world [1, 25]) learning, and we extend it here the autonomous recognition of new objects.”
Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifiers (2018, neutral): “Another set of related works are open set classiﬁcation methods [17], [18], [19], [1], [2].”