Meta-Recognition: The Theory and Practice of Recognition Score Analysis

By Walter J. Scheirer, Anderson de Rezende Rocha, R. Micheals, Terrance E. Boult. 2011

218 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Towards Open Set Deep Networks (2015, methodology): “, we want to add a meta-recognition algorithm [22, 32] to analyze scores and recognize when deep networks are likely incorrect in their assessment.”
BranchyNet: Fast inference via early exiting from deep neural networks (2016, methodology): “Additionally, it may be possible to use a Meta-Recognition algorithm [19, 26] to estimate the characteristics of unseen test samples and adjust T automatically in order to maintain a specified runtime or accuracy goal.”
Recent Advances in Open Set Recognition: A Survey (2018, methodology): “effectively model the tails of the distribution of distances between training observations using the asymptotic theory [100].”
WildFish: A Large Benchmark for Fish Recognition in the Wild (2018, background): “Based on extreme value theory [34], each training category follows a Weibull distribution [4].”
Generative OpenMax for Multi-Class Open Set Classification (2017, background): “Formalization for open space risk is considered as the relative measure of open space compared to the overall measure space: R open = Unknown Space ( Known Space + Unknown Open Space ) Several recent solutions to multi-class open set problem convert it into a score calibration task [17, 23, 27].”
Improving Confidence Estimates for Unfamiliar Examples (2018, unknown): “Other methods aim to reduce conﬁdent errors by detecting failure [8, 10, 55, 51, 44], for example by looking at how close samples are to decision boundaries or estimating whether a test sample comes from the same distribution as training [5, 25, 28, 48, 22].”
Extreme Value Theory for Open Set Classification - GPD and GEV Classifiers (2018, methodology): “The main reason for this success is the fact that the tails of the distribution of distances between training observations can be modeled effectively using the asymptotic theory provided by EVT [15].”
Probability Models for Open Set Recognition (2014, methodology): “With respect to statistical modeling, the prior EVT approaches [25], [24] only fit on the “negative” side of the decision space and use a statistical hypothesis test”
The Extreme Value Machine (2015, methodology): “The EVM is a scalable nonlinear classifier, with radial inclusion functions that are in some respects similar to RBF kernels, but unlike RBF kernels assume variable bandwidths and skew that are data derived and grounded in EVT.”
OpenGAN: Open-Set Recognition via Open Data Generation (2021, unknown): “OpenMax [5] fits logits to Weibull distributions [46] that recalibrate softmax outputs for open-set recognition.”
Heterogeneous information fusion: A novel fusion paradigm for biometric systems (2011, unknown): “[29] proposed to fit the cohort scores with a generalized extreme value distribution and assess y with respect to this distribution.”
Multi-attribute spaces: Calibration for attribute fusion and similarity search (2012, methodology): “However, the distribution of scores around 0 is much more informative and also more constrained – in fact, according to the statistical Extreme Value Theory (EVT) [16], if scores are bounded from above and below, it must be the Weibull distribution, which has shape parameter k > 0 and scale paramete”
Multi-class Open Set Recognition Using Probability of Inclusion (2014, methodology): “recognition [44,43,42,22,21] simply recommends choosing a tail size as an arbitrary percentage (not exceeding 50%) of the available data.”
Predicting Failures of Vision Systems (2014, background): “Meta-recognition: Inspired by meta-cognition " knowing about knowing " [18], Scheirer et al. [50] coined the term " meta-recognition " for performance prediction methods that analyze post-recognition scores [56].”
Towards Transparent Systems: Semantic Characterization of Failure Modes (2014, background): “Predicting failure: Methods that predict overall performance of a system on a collection of test images by analyzing statistics of the test data or postrecognition scores [10–15] are not applicable to our goal of identifying specific failure modes of the system, and semantically characterizing them.”