Face X-Ray for More General Face Forgery Detection

By Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, Baining Guo. Computer Vision and Pattern Recognition, 2019

1,221 citations recorded by the source corpus.

In this paper we propose a novel image representation called face X-ray for detecting forgery in face images. The face X-ray of an input face image is a greyscale image that reveals whether the input image can be decomposed into the blending of two images from different sources. It does so by showing the blending boundary for a forged image and the absence of blending for a real image. We observe that most existing face manipulation methods share a common step: blending the altered face into an existing background image. For this reason, face X-ray provides an effective way for detecting forgery generated by most existing face manipulation algorithms. Face X-ray is general in the sense that it only assumes the existence of a blending step and does not rely on any knowledge of the artifacts associated with a specific face manipulation technique. Indeed, the algorithm for computing face X-ray can be trained without fake images generated by any of the state-of-the-art face manipulation methods. Extensive experiments show that face X-ray remains effective when applied to forgery generated by unseen face manipulation techniques, while most existing face forgery detection or deepfake detection algorithms experience a significant performance drop.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d4e54ea33c097cc0a741200d45053e78f5d5ba9c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Video Transformer for Deepfake Detection with Incremental Learning (2021, background): “A wide variety of detection systems [1, 3, 13, 14, 27, 30, 54] employ CNNs and recurrent networks to detect manipulated media.”
Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues (2020, methodology): “Following previous face forgery detection methods [51,19,40,5], we conduct our experiments on the challenging FaceForensics++ [50] dataset.”
UCF: Uncovering Common Features for Generalizable Deepfake Detection (2023, methodology): “Face Xray [20] uses blended artifacts in forgeries to achieve generalization.”
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection (2020, methodology): “A particularly effective method is Face X-ray [35], which proposes to predict the blending boundary between the background image and the inserted, altered face.”
Creating, Using, Misusing, and Detecting Deep Fakes (2022, background): “Because this blending introduces a slight artifact along the facial boundary, a neural network can be trained to detect artifacts at the boundary of a stitched face (L. Li et al. 2019).”
Learning on Gradients: Generalized Artifacts Representation for GAN-Generated Images Detection (2023, background): “They mainly depend on local regions artifacts [4, 44], blending boundary [26], global textures [30], and frequency-level artifacts [13, 18, 19, 45].”
Local Relation Learning for Face Forgery Detection (2021, neutral): “Face X-ray (Li et al. 2020a) localizes the forged boundary in a self-supervised mechanism, achieving remarkable performance at high-quality data.”
FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset (2021, methodology): “In particular, the detection performance of Face X-ray, F3Net, and LipForensics are 53.5%, 59.8%, and 50.4%, respectively.”
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain (2021, methodology): “Face X-ray [22] mainly focuses on the blending step which exists in most face forgery and thus achieved state-of-the-art performance on transferability in raw videos.”
Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection (2021, background): “Face X-ray [24] innovatively uses self-generated data to train the network to locate blending boundaries, which greatly improves the generalization ability.”
Media Forensics and DeepFakes: An Overview (2020, neutral): “pletely different perspective is followed in [199].”
Generalizing Face Forgery Detection with High-frequency Features (2021, methodology): “We share a similar intuition to [28], which focuses on the blending operation and detects the blending boundary.”