WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation

By Jongheon Jeong, Yang Zou, Taewan Kim, Dongqing Zhang, Avinash Ravichandran, Onkar Dabeer. Computer Vision and Pattern Recognition, 2023

563 citations recorded by the source corpus.

Visual anomaly classification and segmentation are vital for automating industrial quality inspection. The focus of prior research in the field has been on training custom models for each quality inspection task, which requires task-specific images and annotation. In this paper we move away from this regime, addressing zero-shot and few-normal-shot anomaly classification and segmentation. Recently CLIP, a vision-language model, has shown revolutionary generality with competitive zero-/few-shot performance in comparison to full-supervision. But CLIP falls short on anomaly classification and segmentation tasks. Hence, we propose window-based CLIP (WinCLIP) with (1) a compositional ensemble on state words and prompt templates and (2) efficient extraction and aggregation of window/patch/image-level features aligned with text. We also propose its few-normal-shot extension Win-CLIP+, which uses complementary information from normal images. In MVTec-AD (and VisA), without further tuning, WinCLIP achieves 91.8%/85.1% (78.1%/79.6%) AU-ROC in zero-shot anomaly classification and segmentation while WinCLIP + does 93.1%/95.2% (83.8%/96.4%) in 1-normal-shot, surpassing state-of-the-art by large margins.

Subfields: Artificial Intelligence, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/aa207668318fec38d60b79f407fb64982e46fce9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

AnomalyGPT: Detecting Industrial Anomalies using Large Vision-Language Models (2023, methodology): “The design of the decoder is primarily inspired by PatchCore [23], WinCLIP [11], and APRIL-GAN [2].”
Deep Industrial Image Anomaly Detection: A Survey (2023, unknown): “WinCLIP [144] utilizes another large model called CLIP [145] for ZSAD.”
AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection (2023, unknown): “This helps avoid the need for manually defined text prompt templates that require extensive engineering (Jeong et al., 2023).”