Detecting and Simulating Artifacts in GAN Fake Images

By Xu Zhang, Svebor Karaman, Shih-Fu Chang. 2019

652 citations recorded by the source corpus.

To detect GAN generated images, conventional supervised machine learning algorithms require collecting a large number of real images as well as fake images generated by the targeted GAN model. However, the specific model used by the attacker is often unavailable. To address this, we propose a GAN simulator, AutoGAN, which can simulate the artifacts produced by the common pipeline shared by several popular GAN models. Additionally, we identify a unique artifact caused by the up-sampling component included in the common GAN pipelines. We show theoretically such artifacts are manifested as replications of spectra in the frequency domain and thus propose a classifier model based on the spectrum input, rather than the pixel input. By using the simulated images to train a spectrum based classifier, even without seeing the fake images produced by the targeted GAN model during training, our approach achieves state-of-the-art performances on detecting fake images generated by popular GAN models such as CycleGAN.

Subfields: Computer Vision and Pattern Recognition, Generative Models, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

In Safety, Robustness & Interpretability

Citations in the corpus

CNN-Generated Images Are Surprisingly Easy to Spot… for Now (2019, methodology): “[50], we visualize the average frequency spectra from each dataset to study the artifacts generated by CNNs, as shown in Figure 7.”
Analyzing and Improving the Image Quality of StyleGAN (2019, background): “At present, classifier-based methods can quite reliably detect generated images, regardless of their exact origin [24, 39, 35, 45, 36].”
Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, methodology): “keypoints computed from the frequency domain, instead of employing raw sample pixels, was introduced in [168].”
Focal Frequency Loss for Image Reconstruction and Synthesis (2020, background): “Recent studies [59, 72, 22] in media forensics have shown some notable periodic patterns in the frequency spectra of manipulated images, which may be consistent with artifacts in the spatial domain.”
Fake news, disinformation and misinformation in social media: a review (2023, neutral): “…layers in neural networks place data in a hierarchy of different concepts, which ultimately learn from their own mistakes (Parrish Kaliyar et al. 2020; Zhang et al. 2019a; Deepak and Chitturi 2020; Shu et al. 2018a; Wang et al. 2019c) prefer to combine traditional machine learning and deep…”
ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection (2018, background): “Some papers have focused their effort in the detection of manipulations on faces [3, 20] or GAN generated images [35, 53].”
Ensembling with Deep Generative Views (2021, background): “that despite their rapid improvement, the generative models do, in fact, still exhibit artifacts, exploitable for reliable detection of GAN-generated imagery [59, 10, 62, 66, 16].”
The Stable Signature: Rooting Watermarks in Latent Diffusion Models (2023, methodology): “Other approaches use traces left by the generators in the spatial [54, 97] or frequency [26, 106] domains.”
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection (2020, background): “Another line of research relates to the observation that fully CNN-generated images may exhibit anomalous patterns in the frequency spectrum [19, 18, 63, 45].”
Local Relation Learning for Face Forgery Detection (2021, background): “As stated in (Zhang, Karaman, and Chang 2019), most existing face manipulated methods are based on GAN, where the up-sampling operation causes anomalies in the frequency statistical properties of the forged faces.”
Media Forensics and DeepFakes: An Overview (2020, background): “In [174], instead, a GAN simulator is proposed to reproduce common GAN-image artifacts, which manifest as spectral peaks in the Fourier domain.”
Towards Universal Fake Image Detectors that Generalize Across Generative Models (2023, methodology): “Earlier methods studied whether one can even learn a classifier that can detect other images from the same generative model [25, 33, 46], and later work found that such classifiers do not generalize to detecting fakes from other models [19, 52].”
Two-branch Recurrent Network for Isolating Deepfakes in Videos (2020, background): “[69] show that image manipulations leave mediumand high-frequency traces, and since most of the deepfakes methods reconstruct a face with an average pixel-wise `2 loss (Mean Square Error (MSE)), producing somewhat blurry (low-frequency) facial features, and because of the up-sampling commonly used i”
A review of ensemble learning and data augmentation models for class imbalanced problems: combination, implementation and evaluation (2023, background): “Moreover, GANs [330, 253] and re-lated data generation methods such as deepfake [331, 332, 333] have negative connotations since they can be used to imperson-ate individuals, by generating images, speech and related bio-metrics [334].”
Detection of AI-Generated Synthetic Faces (2022, methodology): “In this analysis, the following synthetic image detectors are considered: Xception [35], SRNet [5], Spec [61], M-Gb [56], Co-Net [42], Wang2020 [54], PatchForensics [8].”
DIRE for Diffusion-Generated Image Detection (2023, methodology): “[51] propose a GAN simulator, AutoGAN, to simulate the artifacts produced by standard GAN pipelines.”
Global Texture Enhancement for Fake Face Detection in the Wild (2020, background): “Recently, some researchers have investigated the problem of fake face detection [17, 26, 27, 23, 24, 32, 34, 30].”
On The Detection of Synthetic Images Generated by Diffusion Models (2022, methodology): “We compare the following detectors: Spec [9], based on frequency analysis, PatchForensics [18], that relies on local patch analysis, Wang2020 [15], a ResNet50 with blurring and compression augmentation, and Grag2021 [16], same backbone as before but avoiding down-sampling in the first layer and inte”
Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues (2020, methodology): “Some studies use Wavelet Transform (WT) [7] or Discrete Fourier Transform (DFT) [59,57,19] to convert pictures to frequency domain and mine underlying artifacts.”
GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image (2023, background): “4 GenImage Benchmark”