Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues

By Yuyang Qian, Guojun Yin, Lu Sheng, Jing Shao. European Conference on Computer Vision, 2020

1,166 citations recorded by the source corpus.

As realistic facial manipulation technologies have achieved remarkable progress, social concerns about potential malicious abuse of these technologies bring out an emerging research topic of face forgery detection. However, it is extremely challenging since recent advances are able to forge faces beyond the perception ability of human eyes, especially in compressed images and videos. We find that mining forgery patterns with the awareness of frequency could be a cure, as frequency provides a complementary viewpoint where either subtle forgery artifacts or compression errors could be well described. To introduce frequency into the face forgery detection, we propose a novel Frequency in Face Forgery Network (F3-Net), taking advantages of two different but complementary frequency-aware clues, 1) frequency-aware decomposed image components, and 2) local frequency statistics, to deeply mine the forgery patterns via our two-stream collaborative learning framework. We apply DCT as the applied frequency-domain transformation. Through comprehensive studies, we show that the proposed F3-Net significantly outperforms competing state-of-the-art methods on all compression qualities in the challenging FaceForensics++ dataset, especially wins a big lead upon low-quality media.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/4f899299c366fa1b465afa109412b4597448e554

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes (2021, background): “Recent work using multi-frame inputs [50, 40] and video architecture [48] have shown the competitive potential of leveraging temporal information.”
UCF: Uncovering Common Features for Generalizable Deepfake Detection (2023, methodology): “To assess the generalization capacity of our framework, we reproduce ten competing methods under consistent conditions for a comprehensive comparison: Xception [37], Face X-ray [23], F3Net [36], SRM [30], SPSL [28], RECCE [6], CORE [35] , SLADD [7], and Liang et al. [27].”
Detecting Deepfakes with Self-Blended Images (2022, background): “Some studies [23, 37, 43, 44, 50] focus on the frequency domain to capture forgery traces more effectively.”
Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Domain Learning (2024, methodology): “Recently, prior investigations (Frank et al. 2020; Durall et al. 2020) have substantiated the efficacy of frequency artifacts in the realm of deepfake detection, notably in the context of facial detection (Qian et al. 2020; Luo et al. 2021).”
End-to-End Reconstruction-Classification Learning for Face Forgery Detection (2022, methodology): “Since few existing arts report the performance on it, we re-implement several state-of-the-art methods for a fair comparison, including RFM [45], Add-Net [60], F 3 -Net [33] and MultiAtt [55].”
Local Relation Learning for Face Forgery Detection (2021, positive): “…a novel perspective of face forgery detection via local relation learning to focuses on the relation of local regions, which is a generalized descriptor that can be used to effectively capture the forged trace, such as abnormal texture (Liu et al. 2020) and high frequency noise (Qian et al. 2020).”
MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment (2022, background): “On account of the mechanism of GAN, some unreal textures are lifelike, which are hard to distinguish by human eyes but easy for machines to perceive [33].”
Learning on Gradients: Generalized Artifacts Representation for GAN-Generated Images Detection (2023, background): “F (3)Net [34] designs a two-stream collaborative learning network to mine the forgery patterns in fake images, which exploits two frequency-aware clues, frequency-aware decomposed image components and local frequency statistics, to present subtle forgery patterns and high-level semantics, respective”
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection (2020, background): “Another line of research relates to the observation that fully CNN-generated images may exhibit anomalous patterns in the frequency spectrum [19, 18, 63, 45].”
Detection of AI-Generated Synthetic Faces (2022, background): “Frequency-aware features are learned in the DCT domain in [47] to exploit both local and global frequency clues.”
Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection (2022, methodology): “To ensure fair comparisons, we use the provided codes of Xception [41], RFM [47], and SRM [30] from the authors, and we reimplement Face X-ray [24] and F3Net [39] rigorously following the companion paper’s instructions and train these models under the same settings.”
FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset (2021, methodology): “F3Net [66] This deepfake detection method achieves the state-of-the-art performance by employing frequency-aware clues and local frequency statistic for deepfake detection in frequency domain.”
GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image (2023, background): “F3Net [23] contains two branches, namely Frequency-aware Image Decomposition (FAD) and Local Frequency Statistics (LFS).”
DIRE for Diffusion-Generated Image Detection (2023, methodology): “We evaluate the performance of our DIRE, CNNDet [47], and F3Net [34] trained on the ADM subset of the LSUN-Bedroom split.”
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain (2021, methodology): “Following recent related works [36, 29, 23, 24] of face forgery detection, we conduct our experiments on the three benchmark public deepfake datasets: FaceForensics++(FF++) [38], Celeb-DF [27] and DFDC [10].”