In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking

By Yuezun Li, Ming-Ching Chang, Siwei Lyu. International Workshop on Information Forensics and Security, 2018

898 citations recorded by the source corpus.

The new developments in deep generative networks have significantly improve the quality and efficiency in generating realistically-looking fake face videos. In this work, we describe a new method to expose fake face videos generated with deep neural network models. Our method is based on detection of eye blinking in the videos, which is a physiological signal that is not well presented in the synthesized fake videos. Our method is evaluated over benchmarks of eye-blinking detection datasets and shows promising performance on detecting videos generated with DNN based software DeepFake.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/47364dceef11cb6f36af8b68b127b74da45a0978

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection (2020, background): “datasets [46, 38, 30, 16], it is possible to train deep convolutional neural networks (CNNs) to detect fake videos [3, 25, 64, 4, 14, 45, 42, 36].”
Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos (2019, methodology): “Most of them were evaluated using only one or a few databases, including the CGvsPhoto database [25], the Deepfakes databases [2, 16, 17], and the FaceForensics/FaceForensics++ databases [26, 27].”
Source Generator Attribution via Inversion (2019, methodology): “A related problem is the detection of whether a given image was generated by any GAN-type generator, which has successfully been demonstrated by several groups using various meth-ods [18, 22, 7, 6, 14, 20].”
FaceForensics++: Learning to Detect Manipulated Facial Images (2019, methodology): “While DeepFakes is also training one model per manipulation, it uses a fixed post-processing pipeline sim- ilar to the computer-based manipulation methods and thus has consistent artifacts.”
Deep Learning for Deepfakes Creation and Detection (2019, methodology): “To discriminate real and fake videos, Li et al. [78] ﬁrst decompose the videos into frames where face regions and then eye areas are extracted based on six eye landmarks.”
Two-branch Recurrent Network for Isolating Deepfakes in Videos (2020, methodology): “[36] used the observation that initial versions of deepfakes were not blinking.”
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain (2021, methodology): “Some methods for deepfake detection focus on hand-crafted facial features from the video, such as eye blinking [24], inconsistent head poses [48], facial expression change [2].”
A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials (2022, methodology): “RealSource DeepfakeSource Continual Deepfake-TIMIT[40] VidTIMITdataset[66] KnownDeepfaketech ✗ UADFW[80] EBVdataset[44] KnownDeepfaketech ✗ FaceForensics++[64] YouTube KnownDeepfaketech ✗ Celab-DFv2[47] YouTube KnownDeepfaketech ✗ DFDC[19] Actors KnownDeepfaketech ✗ WildDeepfake[89] Internet…”
The Creation and Detection of Deepfakes (2020, background): “Regarding deepfakes, [30] monitored blood volume paerns (pulse) under the skin, and [97] took a more robust approach by monitoring irregular eye blinking paerns.”
Exploring Temporal Coherence for More General Video Face Forgery Detection (2021, background): “[33] introduce leveraging eye blinking for detecting generated fake face videos.”