ManTra-Net: Manipulation Tracing Network for Detection and Localization of Image Forgeries With Anomalous Features

By Yue Wu, Wael AbdAlmageed, Premkumar Natarajan. Computer Vision and Pattern Recognition, 2019

655 citations recorded by the source corpus.

To fight against real-life image forgery, which commonly involves different types and combined manipulations, we propose a unified deep neural architecture called ManTra-Net. Unlike many existing solutions, ManTra-Net is an end-to-end network that performs both detection and localization without extra preprocessing and postprocessing. \manifold{} is a fully convolutional network and handles images of arbitrary sizes and many known forgery types such splicing, copy-move, removal, enhancement, and even unknown types. This paper has three salient contributions. We design a simple yet effective self-supervised learning task to learn robust image manipulation traces from classifying 385 image manipulation types. Further, we formulate the forgery localization problem as a local anomaly detection problem, design a Z-score feature to capture local anomaly, and propose a novel long short-term memory solution to assess local anomalies. Finally, we carefully conduct ablation experiments to systematically optimize the proposed network design. Our extensive experimental results demonstrate the generalizability, robustness and superiority of ManTra-Net, not only in single types of manipulations/forgeries, but also in their complicated combinations.

Subfields: Biophysics, Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/f4d38c99b8064efba89adddcfac91c85b9e687e9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization (2020, methodology): “ not always be known in advance and a single image may contain multiple types of manipulations. Therefore, it is important to have systems that detect general types of manipulations. Some recent work [37,43,4,4] has addressed this problem, and shown success in building models that have robustness to”
Recurrent Convolutional Strategies for Face Manipulation Detection in Videos (2019, neutral): “Copy-move and splicing detection datasets and methods are abundant for images [31, 45, 44, 46] and less dominant in videos [20].”
Exploring Temporal Coherence for More General Video Face Forgery Detection (2021, background): “Some recent studies [8, 4, 46, 14, 27, 56] aim to locate the visual artifacts in the forged images and make predictions based on the location results.”
Media Forensics and DeepFakes: An Overview (2020, methodology): “Fixed high-pass filters in the first layer have been used also more recently in [141], and in combination with standard filters in [142].”
MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection (2021, methodology): “Feature concatenation at an early stage is adopted by Mantra-Net [15], SPAN [18] and JPEG-ComNet [27], while CAT-Net [28] concatenates the features at a middle stage.”
Images and Misinformation in Political Groups: Evidence from WhatsApp in India (2020, methodology): “We do not observe any significant differences in accuracy for detecting manipulated images or memes, even using features specific to those images, whether features from a recent paper that detects potentially manipulated pixels (Wu et al. [2019]) or features about text in the image (see Methods).”