Exposing DeepFake Videos By Detecting Face Warping Artifacts

By Yuezun Li, Siwei Lyu. CVPR Workshops, 2018

1,204 citations recorded by the source corpus.

In this work, we describe a new deep learning based method that can effectively distinguish AI-generated fake videos (referred to as {\em DeepFake} videos hereafter) from real videos. Our method is based on the observations that current DeepFake algorithm can only generate images of limited resolutions, which need to be further warped to match the original faces in the source video. Such transforms leave distinctive artifacts in the resulting DeepFake videos, and we show that they can be effectively captured by convolutional neural networks (CNNs). Compared to previous methods which use a large amount of real and DeepFake generated images to train CNN classifier, our method does not need DeepFake generated images as negative training examples since we target the artifacts in affine face warping as the distinctive feature to distinguish real and fake images. The advantages of our method are two-fold: (1) Such artifacts can be simulated directly using simple image processing operations on a image to make it as negative example. Since training a DeepFake model to generate negative examples is time-consuming and resource-demanding, our method saves a plenty of time and resources in training data collection; (2) Since such artifacts are general existed in DeepFake videos from different sources, our method is more robust compared to others. Our method is evaluated on two sets of DeepFake video datasets for its effectiveness in practice.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2d066beb34469559e0fc5e5ab4d68dc736cfd46f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Generalizing Face Forgery Detection with High-frequency Features (2021, background): “Some works notice the generalization problem and attempt to utilize additional tasks [14, 35, 45, 44] or generate samples capturing the typical defects [30, 28, 54].”
Celeb-DF: A New Dataset for DeepFake Forensics (2019, background): “A recent twist to the disconcerting problem of online dis-information are the DeepFakes – falsiﬁed images or videos with human faces created by AI algorithms, most notably, deep neural networks and generative adversarial networks [8].”
Two-branch Recurrent Network for Isolating Deepfakes in Videos (2020, methodology): “The table also reports the result of a self-supervised method DSP-FWA [37].”
A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials (2022, background): “Along with the discussed benchmarks, many approaches have been proposed for deepfake detection (e.g., [64, 45, 57, 4, 54, 56, 53, 76, 24, 77]).”
Global Texture Enhancement for Fake Face Detection in the Wild (2020, background): “Other works [18, 33] focused on the alignment of face landmarks to check whether the face is edited by face-swapping tools like DeepFakes [19].”
Face X-Ray for More General Face Forgery Detection (2019, background): “FWA [23] also creates negative samples from real images.”
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method using Affective Cues (2020, methodology): “(4) FWA [36] uses a CNN to expose the face warping artifacts introduced by the resizing and interpolation operations.”
Artificial intelligence in the creative industries: a review (2020, methodology): “Analysis Ads/film analysis [65] [65] [65] GP [66] Content retrieval [67–70] [71] [72] PM [73] Fake detection [74,75] [74] Blockchain [76] Recommendation [77] Deep belief net [78] Contrast enhancement [79] [80] Histogram [81].”
Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, methodology): “[160] VGG16, ResNet50, ResNet101, ResNet152 DLib facial landmarks AUC=84.”
Deepfake Video Detection Using Convolutional Vision Transformer (2021, methodology): “Yuezun and Siwei [34] proposed a CNN architecture that takes advantage of the image transform (i.”
Deep learning for deepfakes creation and detection: A survey (2019, methodology): “ResNet50, ResNet101 and ResNet152 [81]. A deep 6 learning method to detect deepfakes based on the artifacts observed during the face warping step of the deepfake generation algorithms was proposed in [82]. The proposed method is evaluated on two deepfake data sets, namely the UADFV and DeepfakeTIMIT”
WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection (2020, background): “Li et al. introduced two different meth-ods [18, 19] to identify possible "artifacts" or eye blinking defects in deepfake videos.”
Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations (2019, methodology): “[26] proposes to train a CNN to detect warping artifacts for the detection of Deepfake videos.”
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain (2021, methodology): “…56.90 VA-MLP [29] 66.40 55.00 VA-LogReg 78.00 55.10 Xception-c40 [37] 95.50 65.50 Multi-task [30] 76.30 54.30 Capsule [31] 96.60 57.50 DSP-FWA [25] 93.00 64.60 SMIL [23] 96.80 56.30 Two-branch [28] 93.20 73.40 F 3 -Net [35] 97.97 65.17 Our method obtains the state-of-the-art performance on…”
FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals (2019, background): “luate the generated faces’ realism [22], [51], [52]. We summarize all of these in Table 1. The methods in the ﬁrst branch investigate the color and noise distributions of speciﬁc networks [53], [54], [55] or speciﬁc people [21], or train CNNs blindly for synthetic images [20], [45], [50]. However, t”
Deep Learning for Deepfakes Creation and Detection (2019, methodology): “Using face [82] VGG16 [80] ResNet50, 101or152[81] Artifacts are discovered using CNN models based on resolution inconsistency between the warped face area and the surrounding context.”