Semantically Coherent Out-of-Distribution Detection

By Jingkang Yang, Haoqi Wang, Litong Feng, Xiaopeng Yan, Huabin Zheng, Wayne Zhang, Ziwei Liu. 2021

160 citations recorded by the source corpus.

Current out-of-distribution (OOD) detection benchmarks are commonly built by defining one dataset as in-distribution (ID) and all others as OOD. However, these benchmarks unfortunately introduce some unwanted and impractical goals, e.g., to perfectly distinguish CIFAR dogs from ImageNet dogs, even though they have the same semantics and negligible covariate shifts. These unrealistic goals will result in an extremely narrow range of model capabilities, greatly limiting their use in real applications. To overcome these drawbacks, we re-design the benchmarks and propose the semantically coherent out-of-distribution detection (SC-OOD). On the SC-OOD benchmarks, existing methods suffer from large performance degradation, suggesting that they are extremely sensitive to low-level discrepancy between data sources while ignoring their inherent semantics. To develop an effective SC-OOD detection approach, we leverage an external unlabeled set and design a concise framework featured by unsupervised dual grouping (UDG) for the joint modeling of ID and OOD data. The proposed UDG can not only enrich the semantic knowledge of the model by exploiting unlabeled data in an unsupervised manner, but also distinguish ID/OOD samples to enhance ID classification and OOD detection tasks simultaneously. Extensive experiments demonstrate that our approach achieves the state-of-the-art performance on SC-OOD benchmarks. Code and benchmarks are provided on our project page: https://jingkang50.github.io/projects/scood.

Subfields: Artificial Intelligence, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Ziwei Liu
2. Han-Jia Ye
3. De-chuan Zhan
4. Da-Wei Zhou
5. Yixuan Li
6. Bo Li
7. Kaiyang Zhou
8. Jingkang Yang

In Radiology, Nuclear Medicine and Imaging

1. Da-Wei Zhou
2. De-chuan Zhan
3. Han-Jia Ye
4. Yixuan Li
5. Ziwei Liu
6. Jingkang Yang
7. Kaiyang Zhou

Citations in the corpus

• Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks (2022, background): “However, real-world applications often face the stream format data [7] and novel classes [8], [9], [10], [11], e.”
• ViM: Out-Of-Distribution with Virtual-logit Matching (2022, methodology): “Different from the aforementioned methods, another line of works tailors the features learned by the network to better identify ID and OOD by imposing dedicated regularization losses [5, 16, 18, 40] or by exposing generated or real collected OOD samples [22, 37].”
• OpenOOD: Benchmarking Generalized Out-of-Distribution Detection (2022, methodology): “By comparing UDG [50] (the best from extra-data part) with KNN [65] (the best from extra data-free part), the advantage of UDG only lies in CIFAR-10 near-OOD, which does not meet the expectation as a large quantity of real outlier data is required.”
• Out-of-distribution Detection with Deep Nearest Neighbors (2022, background): “…& Gales, 2018; Hendrycks et al., 2019; Geifman & El-Yaniv, 2019; Hein et al., 2019; Meinke & Hein, 2019; Mohseni et al., 2020; Liu et al., 2020; Jeong & Kim, 2020; Van Amersfoort et al., 2020; Yang et al., 2021; Chen et al., 2021; Wei et al., 2022; Ming et al., 2022a; Katz-Samuels et al., 2022).”