Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders

By Paul Bergmann, Sindy Löwe, Michael Fauser, David Sattlegger, Carsten Steger. VISIGRAPP, 2018

825 citations recorded by the source corpus.

Convolutional autoencoders have emerged as popular methods for unsupervised defect segmentation on image data. Most commonly, this task is performed by thresholding a pixel-wise reconstruction error based on an $\ell^p$ distance. This procedure, however, leads to large residuals whenever the reconstruction encompasses slight localization inaccuracies around edges. It also fails to reveal defective regions that have been visually altered when intensity values stay roughly consistent. We show that these problems prevent these approaches from being applied to complex real-world scenarios and that it cannot be easily avoided by employing more elaborate architectures such as variational or feature matching autoencoders. We propose to use a perceptual loss function based on structural similarity which examines inter-dependencies between local image regions, taking into account luminance, contrast and structural information, instead of simply comparing single pixel values. It achieves significant performance gains on a challenging real-world dataset of nanofibrous materials and a novel dataset of two woven fabrics over the state of the art approaches for unsupervised defect segmentation that use pixel-wise reconstruction error metrics.

Subfields: Electrical and Electronic Engineering, Industrial and Manufacturing Engineering, Media Technology

View Paper: https://www.semanticscholar.org/paper/9c24454b071bc8e96ea46c5064a7bddf07cca464

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Electrical and Electronic Engineering

In Industrial and Manufacturing Engineering

In Media Technology

Citations in the corpus

MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection (2019, methodology): “[4] pointed out the disadvantages of per-pixel loss functions in autoencoding frameworks when used in anomaly segmentation scenarios and proposed to incorporate spatial information of local patch regions using structural similarity [27] for improved segmentation results.”
CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows (2021, methodology): “These fully-generative models are unable to estimate the exact data likelihoods [6, 24] and do not perform better than the traditional methods [25, 7] according to MVTec survey in [4].”
Surface Defect Detection Methods for Industrial Products: A Review (2021, methodology): “Literature [57] used the SSIM index in traditional image processing as the reconstruc‐ tion loss and introduced it into the AE‐based image reconstruction; tests were carried out on the woven texture data set and the nanofibrous material dataset, both of which ob‐ tained a significant difference c”
Student-Teacher Feature Pyramid Matching for Anomaly Detection (2021, background): “Simliar to typical autoencoders (AE) [7, 30] and variational autoencoders (VAE) [22], CAVGA also suffers from the strong generalization ability which allows good reconstruction for anomalous images.”
Uninformed Students: Student-Teacher Anomaly Detection With Discriminative Latent Embeddings (2019, methodology): “This can result in poor anomaly detection performance due to simple per-pixel comparisons or imperfect reconstructions [8].”
DRÆM – A discriminatively trained reconstruction embedding for surface anomaly detection (2021, methodology): “Many surface anomaly detection methods focus on image reconstruction and detect anomalies based on image reconstruction error [1, 2, 5, 24, 23, 26, 31].”
PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization (2020, neutral): “Type Reconstruction-based methods Embedding similarity based methods Our methods Model AE simm [1], [2], [9] AE L2 [1], [2] VAE Student [2] Patch SVDD [4] SPADE [5] PaDiMR18-Rd100 PaDiMWR50-Rd550 Carpet (87, 64.”
The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection (2021, methodology): “Recently, Bergmann et al. (2019b) pointed out the disadvantages of per-pixel loss functions in autoencoding frameworks when used in anomaly segmentation scenarios and proposed to incorporate spatial information of local patch regions using structural similarity (Wang et al. 2004) for improved…”
Multiresolution Knowledge Distillation for Anomaly Detection (2020, methodology): “SSIM-AE [10] trains an AE with SSIM loss [54] instead of MSE causing to perform just better on defect segmentation.”
Revisiting Reverse Distillation for Anomaly Detection (2023, background): “In early literature, generative models such as autoencoders (AE) [18], generative adversarial networks (GAN) [12], and their variants are used to reconstruct normal images from anomalous ones [1, 2, 6, 31].”
Deep Industrial Image Anomaly Detection: A Survey (2023, methodology): “[68] investigate the influence of Structure Similarity Index Measure (SSIM) and L2 loss on AE reconstruction and anomaly segmentation, providing numerous suggestions for future research.”
Same Same But DifferNet: Semi-Supervised Defect Detection with Normalizing Flows (2020, background): “[5] proposes SSIM-loss to make the reconstruction error dependent on visual similarity.”
Unsupervised Brain Abnormality Detection Using High Fidelity Image Reconstruction Networks (2020, methodology): “Bergmann et al. (Bergmann et al., 2019) previously conﬁrmed signiﬁcant performance gains for an unsupervised segmentation task for nanoﬁbrous materials with use of SSIM.”
EfficientAD: Accurate Visual Anomaly Detection at Millisecond-Level Latencies (2023, methodology): “[12] Paul Bergmann, Sindy Löwe, Michael Fauser, David Sattlegger, and Carsten Steger.”
VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization (2021, methodology): “• Structural Similarity Index - The Structural Similarity Index (SSIM) [30] is used to measure the image similarity by considering visual properties that are lost in the standard MSE approach:”
Patch SVDD: Patch-level SVDD for Anomaly Detection and Segmentation (2020, background): “Further variants have been proposed to utilize structural similarity indices [13], adversarial training [11], negative mining [12], and iterative projection [14].”
Anomaly Detection via Reverse Distillation from One-Class Embedding (2022, methodology): “However, these methods still lack a strong discriminating ability for real-world anomaly detection [3, 5].”