On the Detection of Digital Face Manipulation

By Joel Stehouwer, H. Dang, Feng Liu, Xiaoming Liu, Anil K. Jain. Computer Vision and Pattern Recognition, 2019

759 citations recorded by the source corpus.

Detecting manipulated facial images and videos is an increasingly important topic in digital media forensics. As advanced face synthesis and manipulation methods are made available, new types of fake face representations are being created which have raised significant concerns for their use in social media. Hence, it is crucial to detect manipulated face images and localize manipulated regions. Instead of simply using multi-task learning to simultaneously detect manipulated images and predict the manipulated mask (regions), we propose to utilize an attention mechanism to process and improve the feature maps for the classification task. The learned attention maps highlight the informative regions to further improve the binary classification (genuine face v. fake face), and also visualize the manipulated regions. To enable our study of manipulated face detection and localization, we collect a large-scale database that contains numerous types of facial forgeries. With this dataset, we perform a thorough analysis of data-driven fake face detection. We show that the use of an attention mechanism improves facial forgery detection and manipulated region localization.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/55467ff033b0f08aabc45dce26d42245ae6ef4a4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection (2019, background): “carried out in [9] a complete analysis of different facial manipulation detection methods.”
Generative Adversarial Networks (GANs) (2020, background): “Deepfakes mainly have been explored for facial manipulation which can be classified into three categories, (1) face synthesis is about creating non-existent realistic faces using GANs [216][217]; (2) face swap is about swapping faces [218]; (3) facial attributes and expression [199] is about manipul”
Detecting Deepfakes with Self-Blended Images (2022, methodology): “Most previous detection methods [8,16,26,30,36,48,53, 64] perform well on the in-dataset scenario where they detect forgeries they learned in training; however, some studies [15,21,33,61] have found that the detection performance significantly drops in the cross-dataset scenario where fake Landmark ”
Multi-attentional Deepfake Detection (2021, background): “This is mainly because that, unlike singleattentional network [6] which can use the video-level labels as explicit guidance and be trained in a supervised way, the multi-attentional structure can only be trained in an unsupervised or weakly-supervised way.”
GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image (2023, methodology): “To aid in the development of detectors, early fake image detection dataset [4, 5] primarily concentrate on face forgery, utilizing generative models to manipulate the human face, thereby replacing a person’s identity or modifying the facial attributes.”
UCF: Uncovering Common Features for Generalizable Deepfake Detection (2023, background): “Other notable directions are focusing on some specific representations, i.e., forgery region location [33], neuron behaviors [49], optical flow [4], landmark geometric features [42], 3D decomposition [59], erasing technology [47], and attentional networks [10, 55, 48].”
Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection (2021, background): “[10] introduce location information of manipulated regions to guide the network to focus on key regions.”
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection (2020, background): “datasets [46, 38, 30, 16], it is possible to train deep convolutional neural networks (CNNs) to detect fake videos [3, 25, 64, 4, 14, 45, 42, 36].”
DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection (2020, background): “carried out in [7] a complete analysis of different facial manipulation methods.”
Learning Self-Consistency for Deepfake Detection (2020, background): “Most of these defense methods [60, 26, 37, 51, 46, 5, 23, 21, 6] target detecting suspicious artifacts left in the stitching process, such as eye blinking [26], face warping [27], blending boundaries [23], and fake prototypes [48].”
End-to-End Reconstruction-Classification Learning for Face Forgery Detection (2022, background): “Early face forgery detection methods [1,8,23,31,35,60] typically follow the classic pipeline of learning convolutional neural networks (CNN) for image classification.”
Local Relation Learning for Face Forgery Detection (2021, neutral): “Instead of simply using a multi-task learning strategy, Stehouwer et al. (Stehouwer et al. 2020) highlights the informative regions through an attention mechanism where the attention map is guided by the ground truth manipulation mask.”
The Creation and Detection of Deepfakes (2020, background): “2 [128] 2019 • • • • • CNN+Attention • 299x299 • 3.”
Generalizing Face Forgery Detection with High-frequency Features (2021, methodology): “FFD exploits annotated forgery masks to provide supervision on attention maps.”
Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, background): “[177] Attention Mechanism along with CNN Deep features AUC=99.”
Deep Face Recognition: A Survey (2018, background): “Presentation attack [289], adversarial attack [280], [281], [290], template attack [291] and digital manipulation attack [292], [293] are developing to threaten the security of deep face recognition systems.”
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain (2021, methodology): “To alleviate the risks brought by malicious usage of face forgery, various methods [49, 25, 1, 48, 2, 38, 37, 12, 22, 35, 28, 8, 23] have been proposed.”