Leveraging Frequency Analysis for Deep Fake Image Recognition

By Joel Cameron Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, Thorsten Holz. International Conference on Machine Learning, 2020

943 citations recorded by the source corpus.

Deep neural networks can generate images that are astonishingly realistic, so much so that it is often hard for humans to distinguish them from actual photos. These achievements have been largely made possible by Generative Adversarial Networks (GANs). While deep fake images have been thoroughly investigated in the image domain - a classical approach from the area of image forensics - an analysis in the frequency domain has been missing so far. In this paper, we address this shortcoming and our results reveal that in frequency space, GAN-generated images exhibit severe artifacts that can be easily identified. We perform a comprehensive analysis, showing that these artifacts are consistent across different neural network architectures, data sets, and resolutions. In a further investigation, we demonstrate that these artifacts are caused by upsampling operations found in all current GAN architectures, indicating a structural and fundamental problem in the way images are generated via GANs. Based on this analysis, we demonstrate how the frequency representation can be used to identify deep fake images in an automated way, surpassing state-of-the-art methods.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/322924507dfe691640d1de54f52789e296afbca3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Focal Frequency Loss for Image Reconstruction and Synthesis (2020, background): “The application areas of the frequency domain analysis have been further extended, including media forensics [59, 72, 22, 12], super-resolution [13, 63], generalization analysis [58, 20], magnetic resonance imaging [53], image rescaling [64], etc.”
Detecting Deepfakes with Self-Blended Images (2022, background): “Some studies [23, 37, 43, 44, 50] focus on the frequency domain to capture forgery traces more effectively.”
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection (2020, background): “Another line of research relates to the observation that fully CNN-generated images may exhibit anomalous patterns in the frequency spectrum [19, 18, 63, 45].”
On The Detection of Synthetic Images Generated by Diffusion Models (2022, background): “The presence and distinctiveness of such traces have been proven by extracting a sort of spatial-domain artificial fingerprints [7, 8], but also through frequency-domain analyses showing that the upsampling operation performed in most GAN architectures give rise to clear spectral peaks [9, 10].”
Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Domain Learning (2024, methodology): “This mean spectrum computation involves averaging over 2,000 images, following the methodology detailed in (Frank et al. 2020).”
UCF: Uncovering Common Features for Generalizable Deepfake Detection (2023, background): “To this end, several works [32, 12, 19, 13] utilize frequency information to improve the performance of detectors.”
Media Forensics and DeepFakes: An Overview (2020, background): “A frequency-representation analysis is also described in [173] and shows the presence of artifacts across different neural network architectures, datasets and resolutions.”
Language-Driven Image Style Transfer (2021, methodology): “To mitigate this issue, we can apply techniques from image forensics [88, 89, 90] to detect the authenticity of an image.”
Pros and Cons of GAN Evaluation Measures: New Developments (2021, background): “Image from Frank et al. (2020).”
Ensembling with Deep Generative Views (2021, background): “that despite their rapid improvement, the generative models do, in fact, still exhibit artifacts, exploitable for reliable detection of GAN-generated imagery [59, 10, 62, 66, 16].”
Exploring Temporal Coherence for More General Video Face Forgery Detection (2021, neutral): “frequency, color) [42, 30, 19, 58, 20] or high-level semantics(e.”
The Stable Signature: Rooting Watermarks in Latent Diffusion Models (2023, positive): “Other approaches track traces left by the generators in the spatial [54, 97] or frequency [26, 106] domains.”
Creating, Using, Misusing, and Detecting Deep Fakes (2022, methodology): “Using a similar computational approach, the forensic classifier of Frank et al. (Frank et al. 2020) yields a similarly accurate and generalizable classifier.”
Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data (2020, background): “Most of them rely on low-level visual patterns in GAN-generated images [34, 49, 46, 19, 57] or frequency mismatch [13, 56, 14].”
DIRE for Diffusion-Generated Image Detection (2023, methodology): “[12] present that in the frequency domain, GAN-generated images are more likely to expose severe artifacts mainly caused by upsampling operations in previous GAN architectures.”