DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection

By Rubén Tolosana, Rubén Vera-Rodriguez, Julian Fierrez, Aythami Morales, Javier Ortega-Garcia. 2020

1,115 citations recorded by the source corpus.

The free access to large-scale public databases, together with the fast progress of deep learning techniques, in particular Generative Adversarial Networks, have led to the generation of very realistic fake content with its corresponding implications towards society in this era of fake news. This survey provides a thorough review of techniques for manipulating face images including DeepFake methods, and methods to detect such manipulations. In particular, four types of facial manipulation are reviewed: i) entire face synthesis, ii) identity swap (DeepFakes), iii) attribute manipulation, and iv) expression swap. For each manipulation group, we provide details regarding manipulation techniques, existing public databases, and key benchmarks for technology evaluation of fake detection methods, including a summary of results from those evaluations. Among all the aspects discussed in the survey, we pay special attention to the latest generation of DeepFakes, highlighting its improvements and challenges for fake detection. In addition to the survey information, we also discuss open issues and future trends that should be considered to advance in the field.

Subfields: Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

This Face Does Not Exist... But It Might Be Yours! Identity Leakage in Generative Models (2020, background): “digitally created or manipulated in some way [14], [20], [21], [26].”
Deepfake detection using deep learning methods: A systematic and comprehensive review (2023, methodology): “The adoption of smartphone applications that conduct automated procedures, including lip-syncing, audio instrumentation, and face swaps, has simplified media manipulation (Tolosana et al., 2020).”
Two-branch Recurrent Network for Isolating Deepfakes in Videos (2020, neutral): “For a complete survey, we refer to [38,46] and the recent work in [58,61].”
GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection (2019, background): “ulation [7]–[9]: i) entire face synthesis, ii) identity swap, iii) attribute manipulation, and iv) expression swap.”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “deepFakes [248], backdoor attacks [455].”
FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset (2021, background): “Even though forging and manipulating images or videos has been done in the past [36], generating highly realistic fake human face images [12] or videos [40] and cloning human voices [14] has become much easier and faster than before.”
Deepfake Video Detection Using Convolutional Vision Transformer (2021, methodology): “The face swap process inserts the face of a source image in a target image of which the subject has never appeared [56].”
Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, background): “In [19], an overview of face manipulation and detection techniques was presented.”
Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes (2021, background): “To address this challenge, a series of excellent works has been conducted on detecting deepfakes [61, 41, 1, 49, 33].”
Instance Selection for GANs (2020, background): “Fortunately, much recent effort has been dedicated towards automatic detection of these false images [30].”
Learning Self-Consistency for Deepfake Detection (2020, background): “There are four common types of deepfake [47]: entire image synthesis, modification of facial attribute or expression, and face identity swap.”
Deep learning for deepfakes creation and detection: A survey (2019, methodology): “[32] categorized both creation and detection methods based on the way deepfakes are created, including entire face synthesis, identity swap, attribute manipulation, and expression swap.”
Deep Neural Networks are Surprisingly Reversible: A Baseline for Zero-Shot Inversion (2021, background): “This may inspire future work on detecting deep fake images [94, 95].”
A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild (2020, background): “In addition to the strong positive impact mentioned above, our intention to completely open-source our work is that it can simultaneously also encourage efforts [11, 16, 24, 25] in detecting manipulated video content and their misuse.”