The DeepFake Detection Challenge Dataset

By Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, Cristian Canton-Ferrer. arXiv.org, 2020

323 citations recorded by the source corpus.

Deepfakes are a recent off-the-shelf manipulation technique that allows anyone to swap two identities in a single video. In addition to Deepfakes, a variety of GAN-based face swapping methods have also been published with accompanying code. To counter this emerging threat, we have constructed an extremely large face swap video dataset to enable the training of detection models, and organized the accompanying DeepFake Detection Challenge (DFDC) Kaggle competition. Importantly, all recorded subjects agreed to participate in and have their likenesses modified during the construction of the face-swapped dataset. The DFDC dataset is by far the largest currently and publicly available face swap video dataset, with over 100,000 total clips sourced from 3,426 paid actors, produced with several Deepfake, GAN-based, and non-learned methods. In addition to describing the methods used to construct the dataset, we provide a detailed analysis of the top submissions from the Kaggle contest. We show although Deepfake detection is extremely difficult and still an unsolved problem, a Deepfake detection model trained only on the DFDC can generalize to real "in-the-wild" Deepfake videos, and such a model can be a valuable analysis tool when analyzing potentially Deepfaked videos. Training, validation and testing corpuses can be downloaded from this https URL.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2cdb1b96846609a965496236eaccb54b1790daab

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection (2020, methodology): “Finally, we use 3,215 test set videos from the DeepFake Detection Challenge (DFDC) [16], where subjects were filmed in extreme conditions, such as large poses and low lighting.”
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain (2021, methodology): “In this section, we evaluate the transferability of our method given that it is trained on FF++ with multiple manipulations but tested on Celeb-DF [27] and DFDC [10].”
Deep learning for deepfakes creation and detection: A survey (2019, methodology): “v2 [107], Deepfake Detection Challenge (DFDC) [108], DFDC Preview [109], and DeeperForensics-1.”
Multi-attentional Deepfake Detection (2021, methodology): “To demonstrate the effectiveness of our multi-attentional network, we conduct extensive experiments on different existing datasets, including FaceForensics++[34], CelebDF[25] and DFDC[9].”
Detecting Deepfakes with Self-Blended Images (2022, methodology): “In the cross-dataset evaluation, we train our model on FF++ [53] and evaluate it on CDF [43], DFD [2], DFDC [20], DFDCP [21], and FFIW [68].”
Single-Shot Freestyle Dance Reenactment (2020, background): “The vast majority of the selected target images are taken out of the DFDC dataset [8].”
Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, methodology): “The DFDC database is publicly available on the Kaggle competition [58].”
Towards Measuring Fairness in AI: The Casual Conversations Dataset (2021, methodology): “The DeepFake Detection Challenge (DFDC) [31, 32] provided researchers the opportunity to develop state-of-the-art detectors to tackle fake videos.”
Preserving integrity in online social networks (2020, background): “Accordingly, research on detecting media manipulation has recently become a very prominent topic [30, 123].”
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature (2023, background): “The problem of machine-generated text detection echoes earlier work on detecting deepfakes, artiﬁcial images or videos generated by deep nets, which has spawned substantial efforts in detection of fake visual content (Dolhansky et al., 2020; Zi et al., 2020).”
A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild (2020, background): “In addition to the strong positive impact mentioned above, our intention to completely open-source our work is that it can simultaneously also encourage efforts [11, 16, 24, 25] in detecting manipulated video content and their misuse.”
Exploring Temporal Coherence for More General Video Face Forgery Detection (2021, methodology): “To evaluate the generalization capability of our framework, we test our model on the following datasets: 1) FF++ that contains four types of manipulation as mentioned above; 2) FaceShifter[31](FSh) and 3) DeeperForensics[28](DFo) employ the real videos from FF++ for high-fidelity face swapping; 4) D”
FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset (2021, background): “The videos in the DFDC dataset were captured in different environmental settings and used eight types of synthesizing methods to generate deepfake videos.”