Image Splicing Localization using a Multi-task Fully Convolutional Network (MFCN)

By Ronald Salloum, Yuzhuo Ren, C.-C. Jay Kuo. Journal of Visual Communication and Image Representation, 2017

385 citations recorded by the source corpus.

Abstract In this work, we propose a technique that utilizes a fully convolutional network (FCN) to localize image splicing attacks. We first evaluated a single-task FCN (SFCN) trained only on the surface label. Although the SFCN is shown to provide superior performance over existing methods, it still provides a coarse localization output in certain cases. Therefore, we propose the use of a multi-task FCN (MFCN) that utilizes two output branches for multi-task learning. One branch is used to learn the surface label, while the other branch is used to learn the edge or boundary of the spliced region. We trained the networks using the CASIA v2.0 dataset, and tested the trained models on the CASIA v1.0, Columbia Uncompressed, Carvalho, and the DARPA/NIST Nimble Challenge 2016 SCI datasets. Experiments show that the SFCN and MFCN outperform existing splicing localization algorithms, and that the MFCN can achieve finer localization than the SFCN.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/38f07a1e267d6822b2ac1fce3c5c74f8442043e5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Noiseprint: A CNN-Based Camera Model Fingerprint (2018, neutral): “architectures have been proposed in [53], [54].”
Media Forensics and DeepFakes: An Overview (2020, background): “In [128] a multi-task fully convolutional network is devised, which includes a specific branch to detect boundaries between inserted regions and background and another branch for the surface of the manipulation, while in [129] a segmentation network based”
Local Relation Learning for Face Forgery Detection (2021, background): “Some works (Bappy et al. 2017; Nguyen et al. 2019; Salloum, Ren, and Kuo 2018) use multi-task learning to simultaneously classify the manipulated images and locate the manipulated region.”
Learning Rich Features for Image Manipulation Detection (2018, methodology): “We vary different thresholds and use the highest F1 score as the final score for each image, which follows the same protocol in [33, 29].”
MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection (2021, methodology): “• Same evaluation protocol: MFCN [13] and RGB-N [14] with numbers quoted from the same team [16].”
SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization (2020, methodology): “Manipulation Detection and Localization: Most of existing methods detect and localization a specific type of manipulation only, such as copy-move[7,27,33], removal[44], enhancement[5,6], and splicing[8,16,18,29].”
PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization (2021, background): “In recent years, works [3, 4, 12, 22, 49, 65, 69] are proposed to build generic IMDL models for multiple manipulation types.”
ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection (2018, background): “Other architectures try to exploit even possible boundary artifacts created by the editing process when for example an object is spliced into the image [36, 6].”
Face X-Ray for More General Face Forgery Detection (2019, background): “Subsequently, deep neural network is introduced in image forgery localization, where most works [5, 40, 29, 38] use multi-task learning to simultaneously detect the manipulated images and locate the manipulated region.”
Fighting Fake News: Image Splice Detection via Learned Self-Consistency (2018, methodology): “These learning methods have also been applied to the problem of detecting specific tampering cues, such as double-JPEG compression [17, 18] and contrast enhancement [19].”
ManTra-Net: Manipulation Tracing Network for Detection and Localization of Image Forgeries With Anomalous Features (2019, methodology): “These clues include JPEG compression artifacts [25, 30, 5], edge inconsistencies [39, 53], noise pattern [33, 49, 20], color consistency [21], visual similarity [44, 45, 46], EXIF consistency [28], and camera model [14, 13].”