Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos

By Huy Hoang Le Nguyen, Junichi Yamagishi, Isao Echizen. IEEE International Conference on Acoustics, Speech, and Signal Processing, 2018

803 citations recorded by the source corpus.

Recent advances in media generation techniques have made it easier for attackers to create forged images and videos. State-of-the-art methods enable the real-time creation of a forged version of a single video obtained from a social network. Although numerous methods have been developed for detecting forged images and videos, they are generally targeted at certain domains and quickly become obsolete as new kinds of attacks appear. The method introduced in this paper uses a capsule network to detect various kinds of spoofs, from replay attacks using printed images or recorded videos to computer-generated videos using deep convolutional neural networks. It extends the application of capsule networks beyond their original intention to the solving of inverse graphics problems.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/445def496333e92ac9d7beeb5168a905f7449c4c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain (2021, methodology): “proposed a method [32] which leveraging capsule network [40] to detect face manipulation.”
Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection (2022, methodology): “We further evaluate our method against several state-of-the-art models, including the Two-stream [59], MesoNet [1], Headpose [53], FWA [27], VA-MLP [33], Capsule [35], SMIL [25], Two-branch [32], SPSL [29], MADD [57].”
Video Transformer for Deepfake Detection with Incremental Learning (2021, methodology): “[40] employ a model based on capsule networks to detect manipulated video.”
Generalizing Face Forgery Detection with High-frequency Features (2021, background): “Nguyen et al. [36] introduced the capsule network.”
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics (2019, methodology): “With the escalated concerns over the DeepFakes, there is a surge of interest in developing DeepFakes detection meth-ods recently [6, 17, 27, 53, 33, 28, 41, 40, 35, 34, 36], with an upcoming dedicated global DeepFake Detection Challenge 1 .”
UCF: Uncovering Common Features for Generalizable Deepfake Detection (2023, methodology): “We further evaluate our method against other state-of-the-art models, including the Two-stream [51], MesoNet [3], Headpose [45], FWA [22], VA-MLP [29], Capsule [31], SMIL [21], Two-branch [28], SPSL [25], MADD [48], SLADD [6], CFFs [46], and Schwarcz et al.”
Multi-attentional Deepfake Detection (2021, neutral): “…83.00 53.60 FWA[24] 80.10 56.90 Xception-raw[25] 99.70 48.20 Xception-c23[25] 99.70 65.30 Xception-c40[25] 95.50 65.50 Multi-task[29] 76.30 54.30 Capsule[30] 96.60 57.50 DSP-FWA[24] 93.00 64.60 Two Branch[26] 93.18 73.41 F 3 -Net[33] 98.10 65.17 EfﬁcientNet-B4[39] 99.70 64.29 Ours 99.80 67.44”
Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes (2021, background): “To address this problem, researchers leverage recent advances in deep learning to automatically extract discriminative features for forgery detection [49, 45, 44, 67].”
Deep Learning for Deepfakes Creation and Detection (2019, methodology): “Capsule-forensics [85] - Latent features extracted by VGG-19 network [80] are fed into the capsule network for classiﬁcation.”
The Creation and Detection of Deepfakes (2020, background): “In [106, 119, 131], it was shown that deep neural networks tend to perform beer than traditional image forensic tools on compressed imagery.”
Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos (2019, methodology): “Unlike other single-target methods [22, 11, 7], our proposed method outputs both the probability of an input being spoofed and segmentation maps of the manipulated regions in each frame of the input, as diagrammed in Figure 2.”
End-to-End Reconstruction-Classification Learning for Face Forgery Detection (2022, background): “Early works like [31,35] use state-ofthe-art image classification backbones, e.”
Local Relation Learning for Face Forgery Detection (2021, neutral): “With the tremendous success of deep learning (Chollet 2017; Tan and Le 2019; Lin et al. 2020), some works (Afchar et al. 2018; Nguyen, Yamagishi, and Echizen 2019; R¨ossler et al. 2019) adopt neural networks to automatically extract discriminative features for forgery detection.”
Deepfake detection using deep learning methods: A systematic and comprehensive review (2023, background): “…amount of research has indeed been done on it, including a thorough study and implementation of many common algorithms (Albahar & Almalki, 2019; T. T. Nguyen, Nguyen, et al., 2019; Pashine et al., 2021; Saif & Tehseen, 2022; Shelke & Kasana, 2021; Swathi & Sk, 2021; Weerawardana & Fernando, 2021).”
Deepfake Video Detection Using Convolutional Vision Transformer (2021, background): “[41] proposed a novel deep learning approach to detect forged images and videos.”
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method using Affective Cues (2020, methodology): “1 Capsule [43] 78.”
Protecting World Leaders Against Deep Fakes (2019, background): “Other forensic techniques exploit low-level pixel artifacts introduced during synthesis [17, 1, 21, 24, 32, 13, 25, 19].”
Deep learning for deepfakes creation and detection: A survey (2019, methodology): “Capsuleforensics [106] Capsule networks - Latent features extracted by VGG-19 network [101] are fed into the capsule network for classification.”
Two-branch Recurrent Network for Isolating Deepfakes in Videos (2020, methodology): “3 Capsule [45] 96.”