Detecting Deepfakes with Self-Blended Images

By Kaede Shiohara, Toshihiko Yamasaki. Computer Vision and Pattern Recognition, 2022

588 citations recorded by the source corpus.

In this paper, we present novel synthetic training data called self-blended images (SBIs) to detect deepfakes. SBIs are generated by blending pseudo source and target images from single pristine images, reproducing common forgery artifacts (e.g., blending boundaries and statistical inconsistencies between source and target images). The key idea behind SBIs is that more general and hardly recognizable fake samples encourage classifiers to learn generic and robust representations without overfitting to manipulation-specific artifacts. We compare our approach with state-of-the-art methods on FF++, CDF, DFD, DFDC, DFDCP, and FFIW datasets by following the standard cross-dataset and cross-manipulation protocols. Extensive experiments show that our method improves the model generalization to unknown manipulations and scenes. In particular, on DFDC and DFDCP where existing methods suffer from the domain gap between the training and test sets, our approach outperforms the baseline by 4.90% and 11.78% points in the cross-dataset evaluation, respectively. Code is available at https://github.com/mapooon/SelfBlendedImages.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/ef3b913e6509077c67e678674e2ba33d99a201a5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yanbo Fan
2. Yong Zhang
3. Baoyuan Wu
4. Zhiyuan Yan
5. Chuangchuang Tan
6. Houqiang Li
7. Guanghua Gu
8. Zhendong Wang

In Computer Vision and Pattern Recognition

1. Wen-gang Zhou
2. Hezhen Hu
3. Baoyuan Wu
4. Zhiyuan Yan
5. Chuangchuang Tan
6. Houqiang Li
7. Hong Chen
8. Yanbo Fan

Citations in the corpus

• UCF: Uncovering Common Features for Generalizable Deepfake Detection (2023, background): “These works typically utilize blending artifacts [23, 41] or frequency artifacts [28, 30], and some employ adversarial training to synthesize challenging forgeries [7].”
• Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Domain Learning (2024, methodology): “Furthermore, various methods (Wang et al. 2020, 2021; Chen et al. 2022; Cao et al. 2022; He et al. 2021; Shiohara et al. 2022)”
• DIRE for Diffusion-Generated Image Detection (2023, methodology): “Applying the pre-trained ADM model [10], we generate 50,000 images in total (50 images for each class in ImageNet), i.e ., 40,000 for training, 5,000 for validation, and 5,000 for testing.”