Multi-attentional Deepfake Detection

By Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, Nenghai Yu. Computer Vision and Pattern Recognition, 2021

940 citations recorded by the source corpus.

Face forgery by deepfake is widely spread over the internet and has raised severe societal concerns. Recently, how to detect such forgery contents has become a hot research topic and many deepfake detection methods have been proposed. Most of them model deepfake detection as a vanilla binary classification problem, i.e, first use a backbone network to extract a global feature and then feed it into a binary classifier (real/fake). But since the difference between the real and fake images in this task is often subtle and local, we argue this vanilla solution is not optimal. In this paper, we instead formulate deepfake detection as a fine-grained classification problem and propose a new multi-attentional deepfake detection network. Specifically, it consists of three key components: 1) multiple spatial attention heads to make the network attend to different local parts; 2) textural feature enhancement block to zoom in the subtle artifacts in shallow features; 3) aggregate the low-level textural feature and high-level semantic features guided by the attention maps. Moreover, to address the learning difficulty of this network, we further introduce a new regional independence loss and an attention guided data augmentation strategy. Through extensive experiments on different datasets, we demonstrate the superiority of our method over the vanilla binary classifier counterparts, and achieve state-of-the-art performance. The models will be released recently at https://github.com/yoctta/multiple-attention.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/eb3165d08407275ac518ae6186c082ff7f5276f9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature (2023, background): “While early works in deepfake detection used relatively general-purpose model architectures (Güera & Delp, 2018), many deepfake detection methods rely on the continuous nature of image data to achieve stateof-the-art performance (Zhao et al., 2021; Guarnera et al., 2020), making direct application t”
M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection (2021, methodology): “Training Set Testing Set Xception [48] Multi-task [40] Capsule [41] DSW-FPA [33] Two-Branch [36] F3-Net [44] MaDD [66] DCViT [60] Ours”
Deepfake detection using deep learning methods: A systematic and comprehensive review (2023, methodology): “In addition, Z. Zhao, Zhou et al. (2021) presented a method for detecting face fraud in deepfake videos that is more accurate.”
The Stable Signature: Rooting Watermarks in Latent Diffusion Models (2023, background): “Detection of AI-generated/manipulated images is notably active in the context of deep-fakes [32, 107].”
Detecting Deepfakes with Self-Blended Images (2022, methodology): “Most previous detection methods [8,16,26,30,36,48,53, 64] perform well on the in-dataset scenario where they detect forgeries they learned in training; however, some studies [15,21,33,61] have found that the detection performance significantly drops in the cross-dataset scenario where fake Landmark ”
UCF: Uncovering Common Features for Generalizable Deepfake Detection (2023, methodology): “We further evaluate our method against other state-of-the-art models, including the Two-stream [51], MesoNet [3], Headpose [45], FWA [22], VA-MLP [29], Capsule [31], SMIL [21], Two-branch [28], SPSL [25], MADD [48], SLADD [6], CFFs [46], and Schwarcz et al.”
Learning Self-Consistency for Deepfake Detection (2020, background): “However, binary classifiers are criticized for their interpretability, and several localization methods are introduced through either multi-task learning [37] or attentionbased mechanisms [5, 59].”
Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection (2022, methodology): “We further evaluate our method against several state-of-the-art models, including the Two-stream [59], MesoNet [1], Headpose [53], FWA [27], VA-MLP [33], Capsule [35], SMIL [25], Two-branch [32], SPSL [29], MADD [57].”
End-to-End Reconstruction-Classification Learning for Face Forgery Detection (2022, methodology): “Recent works resort to specific forgery patterns such as noise characteristics [12, 58], local textures [6, 14, 55], and frequency information [22,33] to better detect forgery artifacts that resided in fake faces.”