Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detection

By Davide Cozzolino, Giovanni Poggi, Luisa Verdoliva. Information Hiding and Multimedia Security Workshop, 2017

366 citations recorded by the source corpus.

Local descriptors based on the image noise residual have proven extremely effective for a number of forensic applications, like forgery detection and localization. Nonetheless, motivated by promising results in computer vision, the focus of the research community is now shifting on deep learning. In this paper we show that a class of residual-based descriptors can be actually regarded as a simple constrained convolutional neural network (CNN). Then, by relaxing the constraints, and fine-tuning the net on a relatively small training set, we obtain a significant performance improvement with respect to the conventional detector.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/8b443b98099f4d713dcdc6cd706a7010b457a586

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Media Forensics and DeepFakes: An Overview (2020, background): “A slightly different perspective is considered in [138] where a CNN architecture is built to exactly replicate the behaviour of the original rich-model classifier [119].”
Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues (2020, methodology): “F) [1], Face2Face(F2F) [56], FaceSwap(FS) [2] and NeuralTextures(NT) [55]). The bold results are the best. Methods DF [1] F2F [56] FS [2] NT [55] Steg.Features [24] 67.00% 48.00% 49.00% 56.00% LD-CNN [14] 75.00% 56.00% 51.00% 62.00% Constrained Conv [6] 87.00% 82.00% 74.00% 74.00% CustomPooling CNN ”
Learning Rich Features for Image Manipulation Detection (2018, methodology): “They also combine SRM features by including the quantization and truncation operations with a Convolutional Neural Network (CNN) to perform manipulation localization [8].”
Detecting CNN-Generated Facial Images in Real-World Scenarios (2020, methodology): “Image pre-processing Pre-processing an image before passing it to a CNN-based model is not uncommon in the field of image forgery detection and has been studied by several works [5, 11, 12, 13, 24, 25, 33, 41].”
FaceForensics++: Learning to Detect Manipulated Facial Images (2019, methodology): “We are considering learning-based methods used in the forensic community for generic manipulation detection [9, 16], computer-generated vs natural image detection [49] and face tampering detection [4].”
ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection (2018, methodology): “Bayar16 [7] and Cozzolino17 [11] have been proposed to detect generic manipulations based on a constrained net-”
Detection of GAN-Generated Fake Images over Social Networks (2018, methodology): “Some are methods proposed for the detection of generic image manipulations [3], [24], [25] or specifically designed for computer graphics image detection [13].”
DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection (2020, methodology): “…detection systems were evaluated using the FaceForensics++ database: i) a CNN-based system trained through handcrafted steganalysis features [71], ii) a CNN-based system whose convolution layers are speciﬁcally designed to suppress the high-level content of the image [72], iii) a CNN-based…”
Towards Universal Fake Image Detectors that Generalize Across Generative Models (2023, background): “(iv) Training a classification network where input images are first converted into their corresponding co-occurrence matrices [35] (a technique shown to be effective in image steganalysis and forensics [16,26]), conditioned on which the network predicts the real/fake class.”
Face X-Ray for More General Face Forgery Detection (2019, background): “With the tremendous success of deep learning, some works [10, 33, 7] adopt neural networks to automatically extract discriminative features for forgery detection.”
Detecting GAN generated Fake Images using Co-occurrence Matrices (2019, methodology): “The top results they obtained using a combination of residual features [50, 9] and deep learning [51].”
Two-branch Recurrent Network for Isolating Deepfakes in Videos (2020, neutral): “98 [12] Cozzolino et al.”
Leveraging Frequency Analysis for Deep Fake Image Recognition (2020, methodology): “These traditional methods have recently been expanded by CNN-based methods (Bayar & Stamm, 2016; Bappy et al., 2017; Cozzolino et al., 2017; Zhou et al., 2018), which learn a more complex feature representation, improving the state-of-the-art for tampered media detection.”
DeepFake Detection by Analyzing Convolutional Traces (2020, background): “They addressed the problem of fake detection as a binary classification problem for each frame of manipulated videos, considering different techniques present in the state of the art [1, 4, 7, 8, 10, 34].”
Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints (2018, background): “While recent state-of-the-art visual forensics techniques demonstrate impressive results for detecting fake visual media [16, 53, 27, 13, 22, 11, 35, 67, 68, 26], they have only focused on semantic, physical, or statistical inconsistency of specific forgery scenarios, e.”
Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos (2018, methodology): “[22] implemented a CNN version of this approach.”