Hybrid LSTM and Encoder–Decoder Architecture for Detection of Image Forgeries

By Jawadul H. Bappy, Cody Simons, Lakshmanan Nataraj, Bangalore S. Manjunath, Amit K. Roy-Chowdhury. IEEE Transactions on Image Processing, 2019

422 citations recorded by the source corpus.

With advanced image journaling tools, one can easily alter the semantic meaning of an image by exploiting certain manipulation techniques such as copy clone, object splicing, and removal, which mislead the viewers. In contrast, the identification of these manipulations becomes a very challenging task as manipulated regions are not visually apparent. This paper proposes a high-confidence manipulation localization architecture that utilizes resampling features, long short-term memory (LSTM) cells, and an encoder–decoder network to segment out manipulated regions from non-manipulated ones. Resampling features are used to capture artifacts, such as JPEG quality loss, upsampling, downsampling, rotation, and shearing. The proposed network exploits larger receptive fields (spatial maps) and frequency-domain correlation to analyze the discriminative characteristics between the manipulated and non-manipulated regions by incorporating the encoder and LSTM network. Finally, the decoder network learns the mapping from low-resolution feature maps to pixel-wise predictions for image tamper localization. With the predicted mask provided by the final layer (softmax) of the proposed architecture, end-to-end training is performed to learn the network parameters through back-propagation using the ground-truth masks. Furthermore, a large image splicing dataset is introduced to guide the training process. The proposed method is capable of localizing image manipulations at the pixel level with high precision, which is demonstrated through rigorous experimentation on three diverse datasets.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/a2a9983f60f732a70c2412b396930da188a112f4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization (2020, methodology): “H-LSTM[4] improved this method by introducing a separate encoder-decoder structure to refine the predicted mask, and a Hilbert Curve route to process the image patches, to establish better context information.”
PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization (2021, methodology): “The compared IMDL methods include J-LSTM [21], H-LSTM [24], RGB-N [23], ManTra-Net [11], and SPAN [18] where SPAN has reported the SOTA performance on localization.”
Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos (2019, methodology): “Semantic segmentation methods [18, 5] can also be used for forgery segmentation [7].”
Face X-Ray for More General Face Forgery Detection (2019, background): “[6] present a localization architecture that exploits both frequency domain and spatial context.”
MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection (2021, methodology): “All models TABLE 7 Overall Performance Measured by Com-F1, the Harmonic Mean of Pixel-Level F1 and Image-Level F1, on Five Test Sets Method Columbia CASIAv1+ COVER DEF-12k IMD MEAN H-LSTM 3.8 0.0 0.0 0.4 0.0 0.8 ManrTra-Net 0.0 0.0 0.0 0.0 0.0 0.0 CR-CNN 41.3 30.3 18.1 19.8 23.7 26.6 GSR-Net 4.2 0.0”
Learning on Gradients: Generalized Artifacts Representation for GAN-Generated Images Detection (2023, background): “Several works [1, 11, 12, 31, 32, 34] leverage the spectral distributions of images as the representation of artifacts for fake image detection.”
Media Forensics and DeepFakes: An Overview (2020, background): “solution to detect and localize image manipulations has been proposed in [151].”
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics (2019, neutral): “, [54, 55, 30, 8], can be applied.”