Learning Self-Consistency for Deepfake Detection

By Tianchen Zhao, Xiang Xu, Mingze Xu, Hui Ding, Yuanjun Xiong, Wei Xia. IEEE International Conference on Computer Vision, 2020

391 citations recorded by the source corpus.

We propose a new method to detect deepfake images using the cue of the source feature inconsistency within the forged images. It is based on the hypothesis that images’ distinct source features can be preserved and extracted after going through state-of-the-art deepfake generation processes. We introduce a novel representation learning approach, called pair-wise self-consistency learning (PCL), for training ConvNets to extract these source features and detect deepfake images. It is accompanied by a new image synthesis approach, called inconsistency image genera-tor (I2G), to provide richly annotated training data for PCL. Experimental results on seven popular datasets show that our models improve averaged AUC over the state of the art from 96.45% to 98.05% in the in-dataset evaluation and from 86.03% to 92.18% in the cross-dataset evaluation.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/7649d833eff8f50c972ad2bccb221c6e15b9bf3e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

UCF: Uncovering Common Features for Generalizable Deepfake Detection (2023, methodology): “Face X-ray [20] detects blending boundary artifacts, SPSL [25] proposes a frequency-based method by phase spectrum analysis, LipForensics [14] leverages spatial-temporal networks to identify unnatural mouth movements, SRM [27] utilizes the high-frequency noises for generalizable detection, PCL [49] ”
Deep learning for deepfakes creation and detection: A survey (2019, methodology): “, in [104, 116, 160, 169, 170], but more work needs to be done in this direction.”
Detecting Deepfakes with Self-Blended Images (2022, methodology): “(5) Pair-wise self-consistency learning [66] (PCL) detects deepfakes via measuring consistencies between patches of input images.”
Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection (2022, methodology): “Both these two methods suggest synthesizing new forgeries to augment the training data, where the forgery regions are fixed in the inner faces, and alpha blending is adopted for all samples in [24] while Poisson blending is mainly used in [58].”
Learning on Gradients: Generalized Artifacts Representation for GAN-Generated Images Detection (2023, background): “PCL [46] extracts the distinct source features of”