Media Forensics and DeepFakes: An Overview

By Luisa Verdoliva. IEEE Journal on Selected Topics in Signal Processing, 2020

844 citations recorded by the source corpus.

With the rapid progress in recent years, techniques that generate and manipulate multimedia content can now provide a very advanced level of realism. The boundary between real and synthetic media has become very thin. On the one hand, this opens the door to a series of exciting applications in different fields such as creative arts, advertising, film production, and video games. On the other hand, it poses enormous security threats. Software packages freely available on the web allow any individual, without special skills, to create very realistic fake images and videos. These can be used to manipulate public opinion during elections, commit fraud, discredit or blackmail people. Therefore, there is an urgent need for automated tools capable of detecting false multimedia content and avoiding the spread of dangerous false information. This review paper aims to present an analysis of the methods for visual media integrity verification, that is, the detection of manipulated images and videos. Special emphasis will be placed on the emerging phenomenon of deepfakes, fake media created through deep learning tools, and on modern data-driven forensic methods to fight them. The analysis will help highlight the limits of current forensic tools, the most relevant issues, the upcoming challenges, and suggest future directions for research.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/f4094ab2fd789d09571926868cb2420896126a0f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction (2020, background): “Moreover, the detection of manipulated videos is an active area of research [31, 12, 1, 24, 30, 13, 11, 2, 44] and we foresee significant improvements in this domain.”
MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection (2021, background): “While pictorial content tampering has been long existing, media forensics is a relatively new research field [5].”
Deep learning for deepfakes creation and detection: A survey (2019, methodology): “figures, virtually trying on clothes while shopping, and so on [19, 20].”
Deepfake detection using deep learning methods: A systematic and comprehensive review (2023, background): “For this reason, L. Verdoliva (2020) examined approaches for visual media integrity verification or identifying altered pictures and videos.”
On The Detection of Synthetic Images Generated by Diffusion Models (2022, background): “These traces are different from those typical of modern digital devices enabling fake image detection [6].”
Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, background): “In [12], the main focus was on generic image manipulation and multimedia forensic techniques.”
GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection (2019, methodology): “The most popular methods to generate fake face content can be categorised into four groups, regarding the level of manipulation [7], [8], [9]: i) entire face synthesis, ii) identity swap, iii) attribute manipulation, and iv) expression swap.”
DeepFake Detection by Analyzing Convolutional Traces (2020, background): “An overview on Media forensics with particular focus on Deepfakes has been recently proposed in [41].”
Detection of AI-Generated Synthetic Faces (2022, background): “, detecting hidden traces in the image, and methods proposed for steganalysis have often shown a great potential also in forensics [52].”
MACE: Mass Concept Erasure in Diffusion Models (2024, background): “These models can generate inappropriate content, such as copyrighted artworks [23, 24, 55, 61], explicit content [22, 59, 72], and deepfakes [38, 70].”
Preserving integrity in online social networks (2020, background): “Accordingly, research on detecting media manipulation has recently become a very prominent topic [30, 123].”
Two-branch Recurrent Network for Isolating Deepfakes in Videos (2020, result): “For a complete survey, we refer to [38,46] and the recent work in [58,61].”
A Statistical Turing Test for Generative Models (2023, background): “See surveys [43, 29] for detailed descriptions of state-of-the-art generation and detection methods in vision.”