Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations

By Falko Matern, Christian Riess, Marc Stamminger. 2019

751 citations recorded by the source corpus.

High quality face editing in videos is a growing concern and spreads distrust in video content. However, upon closer examination, many face editing algorithms exhibit artifacts that resemble classical computer vision issues that stem from face tracking and editing. As a consequence, we wonder how difficult it is to expose artificial faces from current generators? To this end, we review current facial editing methods and several characteristic artifacts from their processin pipelines. We also show that relatively simple visual artifacts can be already quite effective in exposing such manipulations, including Deepfakes and Face2Face. Since the methods are based on visual features, they are easily explicable also to non-technical experts. The methods are easy to implement and offer capabilities for rapid adjustment to new manipulation types with little data available. Despite their simplicity, the methods are able to achieve AUC values of up to 0.866.

Subfields: Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

This Face Does Not Exist... But It Might Be Yours! Identity Leakage in Generative Models (2020, background): “digitally created or manipulated in some way [14], [20], [21], [26].”
Detection of AI-Generated Synthetic Faces (2022, background): “These artifacts are exploited in [39], where simple features are built in order to capture them, such as the correlation between the eyes in suitable color spaces.”
Intermediate Layer Optimization for Inverse Problems using Deep Generative Models (2021, background): “An interesting topic of research is whether existing defences against DeepFakes (Matern et al., 2019; Güera & Delp, 2018; Nguyen et al., 2019; Yang et al., 2020) are robust to images that lie outside of the range of the GAN.”
UCF: Uncovering Common Features for Generalizable Deepfake Detection (2023, methodology): “We further evaluate our method against other state-of-the-art models, including the Two-stream [51], MesoNet [3], Headpose [45], FWA [22], VA-MLP [29], Capsule [31], SMIL [21], Two-branch [28], SPSL [25], MADD [48], SLADD [6], CFFs [46], and Schwarcz et al.”
FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset (2021, methodology): “To perform our experiments, we employed eight different deepfake detection methods to compare ours with other deepfake datasets, which are Capsule [13], HeadPose [14], Visual Artifacts (VA)-MLP/LogReg [51], Xception [36], Meso4 [15], and MesoInception [15].”
Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes (2021, background): “To address this challenge, a series of excellent works has been conducted on detecting deepfakes [61, 41, 1, 49, 33].”
Incremental learning for the detection and classification of GAN-generated images (2019, background): “Some of them exploit specific facial artifacts, like asymmetries in the colour of the eyes, or artifacts arising from an imprecise estimation of the underlying geometry, especially on areas around the nose, the border of the face, and the eyebrows [7].”
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method using Affective Cues (2020, methodology): “(5) VA [38] focuses on capturing visual artifacts in the eyes, teeth and facial contours of synthesized faces.”
Deep Learning for Deepfakes Creation and Detection (2019, methodology): “Eye, teach and facial texture [93] - Exploit facial texture diﬀerences, and missing reﬂections and details in eye and teeth areas of deepfakes.”
FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals (2019, background): “927 AUC Own, NA only [59] & [60], image only [51] 0.”
GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection (2019, methodology): “Finally, the fake detector based on Local Artifacts has proven to be the most robust one, with an average absolute worsening of 4.91% EER.”
Two-branch Recurrent Network for Isolating Deepfakes in Videos (2020, methodology): “They also trained a DCNN though they used as negative samples faces undergoing warping artifacts to simulate the deepfake stitching process [37], while [41] used hand-crafted visual features to amplify artifacts.”