A Less Biased Evaluation of Out-of-distribution Sample Detectors

By Alireza Shafaei, Mark W. Schmidt, James J. Little. 2018

88 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Out-of-Domain Detection for Natural Language Understanding in Dialog Systems (2019, methodology): “Various methods have been proposed to improve the OOD detection performance of neural NLU models [6]–[10], and most of them follow a threshold-based protocol.”
Generalized Out-of-Distribution Detection: A Survey (2021, unknown): “However, as research shows that the performance can be largely affected by the correlations between given and real OOD samples [133].”
OpenGAN: Open-Set Recognition via Open Data Generation (2021, unknown): “Because it is challenging to construct a training set that exhaustively spans the open-world, such a classifier may overfit to the outlier data and not sufficiently generalize [48].”
Anomalous Instance Detection in Deep Learning: A Survey (2020, methodology): “Due to these reasons, there is an immediate need for designing principled benchmarks to reliably evaluate the anomaly detection performance [113, 114].”
Distinction Maximization Loss: Fast, Scalable, Turnkey, and Native Neural Networks Out-of-Distribution Detection simply by Replacing the SoftMax Loss (2019, background): “some “out-of-distribution” samples are indeed available during design, using those examples to validate hyperparameters make the solution overﬁtted to detect this particular type of out-distribution (Shafaei et al., 2019). In the ﬁeld, the system will probably operate under a different/novel/unknown”
ViM: Out-Of-Distribution with Virtual-logit Matching (2022, methodology): “Different from these methods, our method does not require additional OOD data and thus avoids biases towards the introduced OOD samples [31].”
Universal Source-Free Domain Adaptation (2020, background): “Though this problem can be addressed by training the source model on a negative source dataset, a wrong choice of negative data makes the model incapable of rejecting unknown target samples encountered after deployment [42].”
A survey of uncertainty in deep neural networks (2021, background): “• Out-of-domain uncertainty [67], [68], [69], [70] Out-of-domain uncertainty represents the uncertainty related to an input drawn from the subspace of unknown”