In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking

By Yuezun Li, Ming-Ching Chang, Siwei Lyu. arXiv.org, 2018

305 citations recorded by the source corpus.

The new developments in deep generative networks have significantly improve the quality and efficiency in generating realistically-looking fake face videos. In this work, we describe a new method to expose fake face videos generated with neural networks. Our method is based on detection of eye blinking in the videos, which is a physiological signal that is not well presented in the synthesized fake videos. Our method is tested over benchmarks of eye-blinking detection datasets and also show promising performance on detecting videos generated with DeepFake.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/47e8acbabda89b0ece513bd90d0b669d9385fcc3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations (2019, background): “[25] propose to expose fake faces by detecting eyeblinking which according to the authors tends to be missing in Deepfake videos.”
Limits of Deepfake Detection: A Robust Estimation Viewpoint (2019, background): “While several studies use features extracted based on visual artifacts, image quality, lipsync, blinking, or warping for classiﬁcation (and might soon be obsolete) [10], [11], [12], [13], this work gives a generalizable statistical framework with guarantees on its reliability.”
FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals (2019, background): “e last approach we would like to introduce in this domain, which can be considered as the most similar to ours as the only fake detector on videos, exploits blinks to detect inauthentic facial videos [36] as another biological signal. 3. Biological Signals The remote extraction of biological signals”
Deepfake Video Detection through Optical Flow Based CNN (2019, background): “Other approaches consider physical characteristics like the work in [8] where the authors propose a detection of eye blinking to expose generated fake face videos and in [2] where facial expression is modeled in order to distinguish a fake speaking pattern from natural one.”
Realistic Speech-Driven Facial Animation with GANs (2019, background): “This lack of expressions is a clear tell-tale sign of generated videos which is often exploited by systems such as the one proposed in [22], which exposes synthetic faces based on the existence and frequency of blinks.”
FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset (2021, background): “Many researchers have dived into this domain and proposed a variety of different deepfake detection methods [2, 3, 13, 16, 21, 24, 44, 45] create an efficient and usable deepfake detection method, deepfake datasets are required.”
DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection (2020, background): “One of the ﬁrst public databases was UADFV [47].”
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics (2019, methodology): “With the escalated concerns over the DeepFakes, there is a surge of interest in developing DeepFakes detection methods recently [6, 17, 27, 53, 33, 28, 41, 40, 35, 34, 36], with an upcoming dedicated global DeepFake Detection Challenge1.”
Detecting and Simulating Artifacts in GAN Fake Images (2019, background): “In response, the forensics community has been working on detecting such generated content [4], [12], [13], [14].”
Artificial Intelligence, Deepfakes and a Future of Ectypes (2018, methodology): “Research is already available on methods to expose Deepfake videos generated with neural networks (Li et al. 2018).”
Exposing DeepFake Videos By Detecting Face Warping Artifacts (2018, methodology): “We validate our method on a DeepFake video dataset from [3].”
DeepFakes: a New Threat to Face Recognition? Assessment and Detection (2018, methodology): “For instance, it is practically impossible to evaluate the methods proposed in [6] and [21] as their implementations are not yet available.”
Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos (2018, methodology): “’s network [9], for example, is video based and uses temporal information to detect eye blinking.”
Exposing Deep Fakes Using Inconsistent Head Poses (2018, methodology): “AUROC based on videos and frames from UADFV dataset [9]”
Celeb-DF: A New Dataset for DeepFake Forensics (2019, unknown): “A recent twist to the disconcerting problem of online dis-information are the DeepFakes – falsiﬁed images or videos with human faces created by AI algorithms, most notably, deep neural networks and generative adversarial networks [8].”
Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, methodology): “The temporal artifacts involve detecting variation in a person’s behavior [146, 147], physiological signals [135, 148-150], coherence [151, 152], and video frame synchronization [153-156].”