OpenOOD: Benchmarking Generalized Out-of-Distribution Detection
By Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kun Ding, Wen-Hsiao Peng, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun and 6 more. Neural Information Processing Systems, 2022
381 citations recorded by the source corpus.
Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications and has thus been extensively studied, with a plethora of methods developed in the literature. However, the field currently lacks a unified, strictly formulated, and comprehensive benchmark, which often results in unfair comparisons and inconclusive results. From the problem setting perspective, OOD detection is closely related to neighboring fields including anomaly detection (AD), open set recognition (OSR), and model uncertainty, since methods developed for one domain are often applicable to each other. To help the community to improve the evaluation and advance, we build a unified, well-structured codebase called OpenOOD, which implements over 30 methods developed in relevant fields and provides a comprehensive benchmark under the recently proposed generalized OOD detection framework. With a comprehensive comparison of these methods, we are gratified that the field has progressed significantly over the past few years, where both preprocessing methods and the orthogonal post-hoc methods show strong potential.
Subfields: Artificial Intelligence, Control and Systems Engineering
View Paper: https://www.semanticscholar.org/paper/4e9e30f4702f64af5aacbb5791172c5b37510dc3
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Control and Systems Engineering
Citations in the corpus
- ADBench: Anomaly Detection Benchmark (2022, background): “While ADBench focuses on the AD tasks, we note that there are some closely related problems, including out-of-distribution (OOD) detection [182, 183], novelty detection [116, 137], and open-set recognition (OSR) [51, 112].”