Deepfake Video Detection Using Recurrent Neural Networks

By David Guera, Edward J. Delp III. 2018

1,115 citations recorded by the source corpus.

In recent months a machine learning based free software tool has made it easy to create believable face swaps in videos that leaves few traces of manipulation, in what are known as "deepfake" videos. Scenarios where these realistic fake videos are used to create political distress, blackmail someone or fake terrorism events are easily envisioned. This paper proposes a temporal-aware pipeline to automatically detect deepfake videos. Our system uses a convolutional neural network (CNN) to extract frame-level features. These features are then used to train a recurrent neural network (RNN) that learns to classify if a video has been subject to manipulation or not. We evaluate our method against a large set of deepfake videos collected from multiple video websites. We show how our system can achieve competitive results in this task while using a simple architecture.

Subfields: Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Preserving integrity in online social networks (2020, background): “The first is based on learning the traces that media modification methods leave in the resulting footage [38, 69] of the media.”
Intermediate Layer Optimization for Inverse Problems using Deep Generative Models (2021, background): “An interesting topic of research is whether existing defences against Deep-Fakes (Matern et al., 2019; G¨uera & Delp, 2018; Nguyen et al., 2019; Yang et al., 2020) are robust to images that lie outside of the range of the GAN.”
ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection (2018, background): “Some papers have focused their effort in the detection of manipulations on faces [3, 20] or GAN generated images [35, 53].”
FaceForensics++: Learning to Detect Manipulated Facial Images (2019, background): “While DeepFakes is also training one model per manipulation, it uses a fixed post-processing pipeline sim- ilar to the computer-based manipulation methods and thus has consistent artifacts.”
DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection (2020, background): “Thus, they considered a recurrent convolutional network similar to [54], trained in this study end-to-end instead of using a pre-trained model.”
WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection (2020, methodology): “Güera et al. [11] revealed that the frame sequence of deepfake videos have unique characteristics, which differentiates them from unmodified videos.”
Media Forensics and DeepFakes: An Overview (2020, methodology): “effective, they perform face manipulation on a frame-by-frame basis and hence may incorrectly follow the face movements. Several methods have been proposed in the literature to exploit this point. In [187], [188] a convolutional Long Short Term Memory (LSTM) network is used to exploit such dependenc”
Deep Learning for Deepfakes Creation and Detection (2019, methodology): “Intra-frameandtemporal inconsisten-cies[77] CNN and LSTM CNN is employed to extract frame-level features, which are distributed to LSTM to construct sequence descriptor useful for classiﬁcation.”
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature (2023, background): “While early works in deepfake detection used relatively general-purpose model architectures (G¨uera & Delp, 2018), many deepfake detection methods rely on the continuous nature of image data to achieve state-of-the-art performance (Zhao et al., 2021; Guarnera et al., 2020), making direct…”
Exposing DeepFake Videos By Detecting Face Warping Artifacts (2018, methodology): “olution cases of afﬁne warped face, we align faces into multiple scales to increase the data diversity (see Figure2). Compared to training a DeepFake model to generate fake images as negative data in [1,10], which is time-consuming and resource-demanding (˘ 72 hours on a NVIDIA GTX 1 arXiv:1811.0065”
Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations (2019, background): “Güera and Delp [13] propose a recurrent neural network incorporating temporal information to detect Deepfake videos.”
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics (2019, methodology): “, [6, 17, 35, 36]) are data-driven, which directly employ various types of DNNs trained on real and DeepFake videos, not relying on any specific artifact.”
Celeb-DF: A New Dataset for DeepFake Forensics (2019, neutral): “A recent twist to the disconcerting problem of online dis-information are the DeepFakes – falsiﬁed images or videos with human faces created by AI algorithms, most notably, deep neural networks and generative adversarial networks [8].”
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method using Affective Cues (2020, methodology): “3413570 23, 33, 49] and algorithms [9, 26, 36, 38, 42, 43, 49, 50, 57, 61, 64] for deepfake detection.”
Recurrent Convolutional Strategies for Face Manipulation Detection in Videos (2019, methodology): “Recurrent models have been widely used in computer vision for face landmark detection [26], face age progression [42] and even face parsing [27]; nevertheless, to the best of our knowledge they have not been employed before for video manipulation, with the only exception of [13].”
ContraGAN: Contrastive Learning for Conditional Image Generation (2020, neutral): “Besides, it would be good to study the general characteristics of generated samples [63] and how we can distinguish fake images from unknown generative models [64, 65, 66]. in previous papers [71, 2, 7].”
FakeSpotter: A Simple Baseline for Spotting AI-Synthesized Fake Faces (2019, background): “ages, which result the existing ﬁngerprints of GANs failed in identifying fake faces. Our work also aims at proposing a general approach for detecting various GAN-synthesized fake faces. Some studies [20,23,24] research Deep Fakes detection, mostly detecting faceswapping. These work mainly utilize t”
Artificial intelligence in the creative industries: a review (2020, methodology): “Analysis Ads/film analysis [65] [65] [65] GP [66] Content retrieval [67–70] [71] [72] PM [73] Fake detection [74,75] [74] Blockchain [76] Recommendation [77] Deep belief net [78] Contrast enhancement [79] [80] Histogram [81].”