Deepfake Video Detection through Optical Flow Based CNN

By Irene Amerini, Leonardo Galteri, Roberto Caldelli, A. del Bimbo. 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 2019

372 citations recorded by the source corpus.

Recent advances in visual media technology have led to new tools for processing and, above all, generating multimedia contents. In particular, modern AI-based technologies have provided easy-to-use tools to create extremely realistic manipulated videos. Such synthetic videos, named Deep Fakes, may constitute a serious threat to attack the reputation of public subjects or to address the general opinion on a certain event. According to this, being able to individuate this kind of fake information becomes fundamental. In this work, a new forensic technique able to discern between fake and original video sequences is given; unlike other state-of-the-art methods which resorts at single video frames, we propose the adoption of optical flow fields to exploit possible inter-frame dissimilarities. Such a clue is then used as feature to be learned by CNN classifiers. Preliminary results obtained on FaceForensics++ dataset highlight very promising performances.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d42b66341e68a8f932af47eae8204f4d55ab089e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, background): “This method [156] exhibits better deepfake detection performance, however, only initial results have been reported.”
DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection (2020, background): “Finally, in [101], Amerini et al. proposed the adoption of optical ﬂow ﬁelds to exploit possible inter-frame dissimilarities.”
Generalizing Face Forgery Detection with High-frequency Features (2021, methodology): “[29, 34, 55, 9, 7, 32] targeted at specific artifacts such as abnormal eye-blinking frequency [29] or head-pose inconsistency [55].”
Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection (2022, background): “Recent studies have made various attempts for deepfake detection and achieve remarkable success [26, 41, 3, 2, 32, 39, 14, 24, 30, 29, 19, 47, 23, 36, 5].”
Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes (2021, background): “Most forgery detection methods process deepfake videos frame-by-frame, and few explore multi-modal and temporal dynamics [74, 4].”
The Creation and Detection of Deepfakes (2020, positive): “In [25] a classier is trained pairs of sequential frames and in [11] the authors rene the network’s focus by monitoring the frames’ optical ow.”
Detecting Deepfakes with Self-Blended Images (2022, background): “, head pose [62], eye blinking [30, 40], mouth movements [27], neuron behaviors [60], optical flow [9], and steganalysis features [24]).”
UCF: Uncovering Common Features for Generalizable Deepfake Detection (2023, background): “Other notable directions are focusing on some specific representations, i.e., forgery region location [33], neuron behaviors [49], optical flow [4], landmark geometric features [42], 3D decomposition [59], erasing technology [47], and attentional networks [10, 55, 48].”
ContraGAN: Contrastive Learning for Conditional Image Generation (2020, positive): “Besides, it would be good to study the general characteristics of generated samples [63] and how we can distinguish fake images from unknown generative models [64, 65, 66]. in previous papers [71, 2, 7].”
Media Forensics and DeepFakes: An Overview (2020, background): “[190] the optical flow field is estimated to exploit discrepancies in motion across frames, but only some preliminary results are presented.”
FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset (2021, methodology): “Most deepfake detectors [3, 4, 16, 23, 34] use FF++ and DFDC datasets to train their models to detect deepfakes.”
Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues (2020, methodology): “Following previous face forgery detection methods [51,19,40,5], we conduct our experiments on the challenging FaceForensics++ [50] dataset.”