Anomaly Detection via Reverse Distillation from One-Class Embedding

By Hanqiu Deng, Xingyu Li. Computer Vision and Pattern Recognition, 2022

945 citations recorded by the source corpus.

Knowledge distillation (KD) achieves promising results on the challenging problem of unsupervised anomaly detection (AD). The representation discrepancy of anomalies in the teacher-student (T-S) model provides essential evidence for AD. However, using similar or identical architectures to build the teacher and student models in previous studies hinders the diversity of anomalous representations. To tackle this problem, we propose a novel T-S model consisting of a teacher encoder and a student decoder and introduce a simple yet effective “reverse distillation” paradigm accordingly. Instead of receiving raw images directly, the student network takes teacher model's one-class embedding as input and targets to restore the teacher's multi-scale representations. Inherently, knowledge distillation in this study starts from abstract, high-level presentations to low-level features. In addition, we introduce a trainable one-class bottleneck embedding (OCBE) module in our T-S model. The obtained compact embedding effectively preserves essential information on normal patterns, but aban-dons anomaly perturbations. Extensive experimentation on AD and one-class novelty detection benchmarks shows that our method surpasses SOTA performance, demonstrating our proposed approach's effectiveness and generalizability.

Subfields: Artificial Intelligence, Computer Networks and Communications, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/3b3aefbbdb64e5812f133f220b3f129a36a30065

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Nguyen H. Tran
2. Xin Lu
3. Onkar Dabeer
4. Xinyi Le
5. Anh Tuan Nguyen
6. Jiulong Shan
7. Steven Quoc Hung Truong
8. Zhikang Liu

In Computer Networks and Communications

1. Nguyen H. Tran
2. Onkar Dabeer
3. Zhikang Liu
4. Dongqing Zhang
5. Yang Zou
6. Xi Li
7. Yiming Zhou
8. Zilei Wang

In Control and Systems Engineering

1. Zhikang Liu
2. Yiming Zhou
3. Zilei Wang
4. Yuansheng Xu

Citations in the corpus

• A Unified Model for Multi-class Anomaly Detection (2022, methodology): “3) The channel dimensions from stage-1 to stage-4 are respectively C , C/ 2 , C/ 2 , and C , where C is the reduced channel dimension that is 256.”
• DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection (2022, methodology): “In contrast, the upper layers of CNN express global semantic information [9].”
• Deep Industrial Image Anomaly Detection: A Survey (2023, neutral): “RD4AD [14] Cosine Similarity ResNet The paper designs the teacher-student model of reverse distillation in a similar way to reconstruction.”
• SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation (2022, methodology): “Many recent methods are proposed to detect low-level texture anomalies [35], such as scratches and cracks, which are common cases in industrial visual inspection [15,31,34,44].”
• AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection (2023, neutral): “Comparison with SOTA full-shot methods In this section, we are interested in the performance gap between AnomalyCLIP and the recently published SOTA full-shot methods, such as Patch-Core (Roth et al., 2022) and RD4AD (Deng & Li, 2022).”
• SimpleNet: A Simple Network for Image Anomaly Detection and Localization (2023, methodology): “The comparison baselines includes AE-SSIM [3], RIAD [31], DRÆM [30], CutPaste [17], CS-Flow [24], PaDiM [6], RevDist [7] and PatchCore [22].”
• Revisiting Reverse Distillation for Anomaly Detection (2023, methodology): “The proposed framework RD++ for anomaly detection, inspired by the idea of reverse distillation [8], consists of several modifications for the architecture and the loss function.”