Learning Rich Features for Image Manipulation Detection

By Peng Zhou, Xintong Han, Vlad I. Morariu, Larry S. Davis. 2018

759 citations recorded by the source corpus.

Image manipulation detection is different from traditional semantic object detection because it pays more attention to tampering artifacts than to image content, which suggests that richer features need to be learned. We propose a two-stream Faster R-CNN network and train it end-to-end to detect the tampered regions given a manipulated image. One of the two streams is an RGB stream whose purpose is to extract features from the RGB image input to find tampering artifacts like strong contrast difference, unnatural tampered boundaries, and so on. The other is a noise stream that leverages the noise features extracted from a steganalysis rich model filter layer to discover the noise inconsistency between authentic and tampered regions. We then fuse features from the two streams through a bilinear pooling layer to further incorporate spatial co-occurrence of these two modalities. Experiments on four standard image manipulation datasets demonstrate that our two-stream framework outperforms each individual stream, and also achieves state-of-the-art performance compared to alternative methods with robustness to resizing and compression.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Safety, Robustness & Interpretability

Citations in the corpus

CNN-Generated Images Are Surprisingly Easy to Spot… for Now (2019, methodology): “Evaluation Following other recent forensics works [46, 16, 38], we evaluate our model’s performance on each dataset using average precision (AP), since it is a thresholdless, ranking-based score that is not sensitive to the base rate of the real and fake images in the dataset.”
NISP: Pruning Networks Using Neuron Importance Score Propagation (2017, background): “Despite their impressive predictive power on a wide range of tasks [33, 40, 41, 13, 15, 48, 46, 24, 44, 42, 12, 45], the redundancy in the parameterization of deep learning models has been studied and demonstrated [6].”
Fighting Fake News: Image Splice Detection via Learned Self-Consistency (2018, methodology): “In concurrent work, Mayer [24] proposed using a Siamese network to predict whether pairs of image patches have the same camera model — a special case of our metadata consistency model (they also propose using this model for splice detection; while promising, these results are very preliminary).”
DIRE for Diffusion-Generated Image Detection (2023, methodology): “Following previous generated-image detection methods [48, 47, 52], we mainly report accuracy (ACC) and average precision (AP) in our experiments to evaluate the detectors.”
Steering Self-Supervised Feature Learning Beyond Local Pixel Statistics (2020, methodology): “[59] detect image tampering generated using semantic masks.”
Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos (2019, background): “A slightly different segmentation approach is to return the boxes that represent the boundaries of the manipulated regions instead of returning segmentation masks [32].”
ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection (2018, background): “In [44] a two-stream network is proposed that exploits both low-level and high-level features.”
Exploring Temporal Coherence for More General Video Face Forgery Detection (2021, methodology): “Most previous methods [60, 61, 43, 45] are trained for known face manipulation techniques.”
Detecting Photoshopped Faces by Scripting Photoshop (2019, methodology): “[43] propose an object detection model, specifically using steganalysis features to reduce the influence of semantics.”
ManTra-Net: Manipulation Tracing Network for Detection and Localization of Image Forgeries With Anomalous Features (2019, methodology): “We compare the known optimal settings for SRMConv2D from [55] and BayarConv2D from [10] with the classic Conv2D layers, and a combined version of all three, which is simply the feature concatenation as shown in Fig.”
SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization (2020, methodology): “RGB-N [43] proposes a Faster R-CNN [28] based method to detect tampered region; hence, its predictions are limited to the rectangular box.”
Hybrid LSTM and Encoder–Decoder Architecture for Detection of Image Forgeries (2019, background): “of rectangle box. In our case, we eliminate the bounding box which has area under 64. As a result, we observe signiﬁcant improvement in AP score. The AP score rises from 0:825 to 923. The AP score of [91] is 0:934. From the above discussion, we can see that the proposed model achieves comparable res”
FakeSpotter: A Simple Baseline for Spotting AI-Synthesized Fake Faces (2019, methodology): “Recently, some research [6,7,9,18] borrow ideas from the traditional digital image forensics to detect GAN-synthesized fake faces/images.”
Generalizing Face Forgery Detection with High-frequency Features (2021, methodology): “, SRMNet [58], Bayar Conv [10], SSTNet [52], and F3Net [38].”
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics (2019, neutral): “, [54, 55, 30, 8], can be applied.”
DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection (2020, background): “In particular, we refer the reader to [45], [46].”
Media Forensics and DeepFakes: An Overview (2020, background): “Hence, a two-stream network is proposed in [147] and extended in [148].”
Noiseprint: A CNN-Based Camera Model Fingerprint (2018, background): “work on noise residuals is joined with a general purpose deep CNN (ResNet 101 in [52]).”
MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection (2021, methodology): “[14] develop a two-stream Faster R-CNN, coined RGB-N, which takes as input the RGB image and its noise counterpart generated by the spatial”