Exposing Deep Fakes Using Inconsistent Head Poses

By Xin Yang, Yuezun Li, Siwei Lyu. IEEE International Conference on Acoustics, Speech, and Signal Processing, 2018

1,129 citations recorded by the source corpus.

In this paper, we propose a new method to expose AI-generated fake face images or videos (commonly known as the Deep Fakes). Our method is based on the observations that Deep Fakes are created by splicing synthesized face region into the original image, and in doing so, introducing errors that can be revealed when 3D head poses are estimated from the face images. We perform experiments to demonstrate this phenomenon and further develop a classification method based on this cue. Using features based on this cue, an SVM classifier is evaluated using a set of real face images and Deep Fakes.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/ee125fef53038c2b31b3c084844e34efd80785e7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Two-branch Recurrent Network for Isolating Deepfakes in Videos (2020, neutral): “6 HeadPose [67] 47.”
Video Transformer for Deepfake Detection with Incremental Learning (2021, background): “Li et al. [31] employ CNNs to detect face warping artifacts in images from the deepfake datasets [29, 55].”
The DeepFake Detection Challenge (DFDC) Dataset. (2020, neutral): “Furthermore, as we observed that many publicly-released datasets [14, 17, 18, 23, 30] did not guarantee that their subjects were willing participants or agreed to have their faces modified, we solicited video data from 3,426 paid actors and actresses speaking in a variety of settings for roughly 15 ”
UCF: Uncovering Common Features for Generalizable Deepfake Detection (2023, methodology): “We further evaluate our method against other state-of-the-art models, including the Two-stream [51], MesoNet [3], Headpose [45], FWA [22], VA-MLP [29], Capsule [31], SMIL [21], Two-branch [28], SPSL [25], MADD [48], SLADD [6], CFFs [46], and Schwarcz et al.”
Exposing DeepFake Videos By Detecting Face Warping Artifacts (2018, methodology): “HeadPose [34] 89.”
Protecting World Leaders Against Deep Fakes (2019, methodology): “This same team also developed a technique [31] for detecting face-swap deep fakes by leveraging differences in the estimated 3-D head pose as computed from features around the entire face and features in only the central (potentially swapped) facial region.”
Learning Self-Consistency for Deepfake Detection (2020, neutral): “To support the research on detecting the other types of face manipulations, several deepfake datasets are released [57, 19, 41, 10, 28, 8, 15, 61] and countermeasures have been introduced.”
Deep Learning for Deepfakes Creation and Detection (2019, methodology): “The UADFV data set [83] contains 49 real videos and 49 fake videos with 32752 frames in total.”
On the Detection of Digital Face Manipulation (2019, background): “Methods Training data UADFV [58] Celeb-DF [31]”
DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection (2020, methodology): “Celeb-DF [26] also provides a face forgery detection benchmark including seven methods [6, 12, 25, 28, 30, 41, 43] trained and tested on different datasets.”
The Deepfake Detection Challenge (DFDC) Preview Dataset (2019, methodology): “These include the Celeb-DF [4], UADFV [5], DeepFake-TIMIT [6], VTD dataset [7], FaceForensics [8] and its successor FaceForensics++ [9].”
Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, methodology): “[143] introduced an approach to detect the deepfakes by estimating the 3D head position from 2D facial landmarks.”
Detecting Deepfakes with Self-Blended Images (2022, background): “, head pose [62], eye blinking [30, 40], mouth movements [27], neuron behaviors [60], optical flow [9], and steganalysis features [24]).”
Creating, Using, Misusing, and Detecting Deep Fakes (2022, methodology): “Ocular asymmetries caused by the lack of explicit semantic knowledge in image- synthesis pipelines has also been exploited in the analysis of facial feature configurations (Yang et al. 2019).”
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics (2019, methodology): “To date, we have the UADFV dataset [53], the DeepFake-TIMIT dataset (DF-TIMIT) [25], the FaceForenscics++ dataset (FF-DF) [40]2, the Google DeepFake detection dataset (DFD) [15], and the FaceBook DeepFake detection challenge (DFDC) dataset [14].”