FaceForensics++: Learning to Detect Manipulated Facial Images
By Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, Matthias Nießner. 2019
3,187 citations recorded by the source corpus.
The rapid progress in synthetic image generation and manipulation has now come to a point where it raises significant concerns for the implications towards society. At best, this leads to a loss of trust in digital content, but could potentially cause further harm by spreading false information or fake news. This paper examines the realism of state-of-the-art image manipulations, and how difficult it is to detect them, either automatically or by humans. To standardize the evaluation of detection methods, we propose an automated benchmark for facial manipulation detection. In particular, the benchmark is based on Deep-Fakes, Face2Face, FaceSwap and NeuralTextures as prominent representatives for facial manipulations at random compression level and size. The benchmark is publicly available and contains a hidden test set as well as a database of over 1.8 million manipulated images. This dataset is over an order of magnitude larger than comparable, publicly available, forgery datasets. Based on this data, we performed a thorough analysis of data-driven forgery detectors. We show that the use of additional domain-specific knowledge improves forgery detection to unprecedented accuracy, even in the presence of strong compression, and clearly outperforms human observers.
Subfields: Computer Vision and Pattern Recognition, Law
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Xiaogang Wang
- Ming-Hsuan Yang
- Trevor Darrell
- Jitendra Malik
- Andrew Zisserman
- Andrea Vedaldi
In Law
- Luisa Verdoliva
- Siwei Lyu
- Yuezun Li
- Andrew Owens
- Alexei A. Efros
- Shruti Agarwal
- Michael Zollhöfer
- Shahroz Tariq
Citations in the corpus
- HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models (2019, background): “Realizing the constraints of available automated metrics, many generative modeling tasks resort to human evaluation and visual inspection [49, 52, 14].”
- Media Forensics and DeepFakes: An Overview (2020, methodology): “dense network with one hidden layer. The second solution (MesoInception4) instead is based on a variant of the inception module that includes dilated convolutions. However, experiments carried out in [178] clearly show that, in a supervised setting, very deep general-purpose networks [179], [180], [”
- Creating, Using, Misusing, and Detecting Deep Fakes (2022, background): “FaceForensics++ (Rossler et al. 2019) consisting of 1,000 YouTube videos of 1,000 different people, mostly news anchors and video bloggers, from which four types of deep-fake videos are created: faceswap17, FaceSwap18, Face2Face (Justus Thies et al. 2016), and Neural Textures (Thies, Zollhöfer, and…”
- High-Fidelity GAN Inversion for Image Attribute Editing (2021, background): “We show inversion and editing results on a real video [28] in Fig.”
- Generative Adversarial Networks (GANs) (2020, background): “Deepfakes mainly have been explored for facial manipulation which can be classified into three categories, (1) face synthesis is about creating non-existent realistic faces using GANs [216][217]; (2) face swap is about swapping faces [218]; (3) facial attributes and expression [199] is about manipul”
- Deepfacelab: Integrated, flexible and extensible face-swapping framework (2020, methodology): “[15] and Face2Face [20]) taken from FaceForensics++ dataset [19].”
- Celeb-DF: A New Dataset for DeepFake Forensics (2019, methodology): “The FaceForensics++ [18] is a large scale dataset which contains five subsets: FaceSwap, DeepFake, DeepFakeDetection, Face2Face and NeuralTextures.”
- The Creation and Detection of Deepfakes (2020, background): “In [106, 119, 131], it was shown that deep neural networks tend to perform beer than traditional image forensic tools on compressed imagery.”
- Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints (2018, background): “[49, 50] introduced a large-scale face manipulation dataset to benchmark forensics classification and segmentation tasks, and demonstrated superior performance when using additional domain-specific knowledge.”
- Detection of AI-Generated Synthetic Faces (2022, neutral): “adapt well to multimedia forensic tasks [48].”
- Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, methodology): “3 FaceForensics++ One of the most famous datasets for deepfake detection is FaceForensics++ [162].”
- FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals (2019, methodology): “Train Test Video Accuracy Celeb-DF [4] FF++ [3] 83.”
- FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping (2019, methodology): “1 [35] Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner.”
- Local Relation Learning for Face Forgery Detection (2021, neutral): “Following the convention, we adopt the widely-used benchmark dataset FaceForensics++ (FF++) (R ¨ ossler et al. 2019) Implementation.”
- End-to-End Reconstruction-Classification Learning for Face Forgery Detection (2022, methodology): “Here we additionally use FF++ c23 [11] (i.”
- Detecting CNN-Generated Facial Images in Real-World Scenarios (2020, methodology): “[42] shows that humans have trouble detecting CNN-modified images.”
- Detecting Photoshopped Faces by Scripting Photoshop (2019, methodology): “(1) FaceForensics++ [30]: A network trained on face swapping and reenactment data; we use the Xcep-tion [10] model trained on raw video frames.”