Two-Stream Neural Networks for Tampered Face Detection

By Peng Zhou, Xintong Han, Vlad I. Morariu, Larry S. Davis. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017

635 citations recorded by the source corpus.

We propose a two-stream network for face tampering detection. We train GoogLeNet to detect tampering artifacts in a face classification stream, and train a patch based triplet network to leverage features capturing local noise residuals and camera characteristics as a second stream. In addition, we use two different online face swaping applications to create a new dataset that consists of 2010 tampered images, each of which contains a tampered face. We evaluate the proposed two-stream network on our newly collected dataset. Experimental results demonstrate the effectness of our method.

Subfields: Computer Vision and Pattern Recognition, Law

View Paper: https://www.semanticscholar.org/paper/5c04b3178af0cc5f367c833030c118701c210229

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Law

Citations in the corpus

Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos (2019, methodology): “Other approaches to detection include using a constrained convolutional layer [8], using a statistical pooling layer [25], using a two-stream network [31], using a lightweight CNN network [2], and using two cascaded convolutional layers at the bottom of a CNN [23].”
Generalizing Face Forgery Detection with High-frequency Features (2021, methodology): “Unlike [11, 52, 57] that only consider extracting noises from an input image, we further apply these filters to low-level features at multiple scales to compose more abundant and informative features.”
M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection (2021, methodology): “[67] uses a two-stream architecture to capture facial manipulation clues and patch inconsistency separately, while [40] simultaneously identifies forged faces and locate the manipulated regions with multi-task learning.”
Deep learning for deepfakes creation and detection: A survey (2019, methodology): “[121], and the face tampering detection method twostream NN [122].”
FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset (2021, background): “Many researchers have dived into this domain and proposed a variety of different deepfake detection methods [2, 3, 13, 16, 21, 24, 44, 45] create an efficient and usable deepfake detection method, deepfake datasets are required.”
ManTra-Net: Manipulation Tracing Network for Detection and Localization of Image Forgeries With Anomalous Features (2019, neutral): “[54] Artifact 2Br-DNN Patch-Lv N 1”
Learning Self-Consistency for Deepfake Detection (2020, background): “Most of these defense methods [60, 26, 37, 51, 46, 5, 23, 21, 6] target detecting suspicious artifacts left in the stitching process, such as eye blinking [26], face warping [27], blending boundaries [23], and fake prototypes [48].”
Deep Learning for Deepfakes Creation and Detection (2019, methodology): “Performance of the proposed method is compared with other prevalent methods such as the face tampering detection method two-stream NN [84], HeadPose [83], and two deepfake detection MesoNet methods, i.e. Meso-4 and MesoInception-4 [73].”
On the Detection of Digital Face Manipulation (2019, methodology): “[61] collected a dataset with face-swapped images generated by an iOS app and an opensource software.”
DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection (2020, methodology): “Celeb-DF [26] also provides a face forgery detection benchmark including seven methods [6, 12, 25, 28, 30, 41, 43] trained and tested on different datasets.”
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection (2020, background): “datasets [46, 38, 30, 16], it is possible to train deep convolutional neural networks (CNNs) to detect fake videos [3, 25, 64, 4, 14, 45, 42, 36].”
Recurrent Convolutional Strategies for Face Manipulation Detection in Videos (2019, background): “Regrettably, compelling datasets for face manipulation detection and evaluation in videos had been lacking in the community; some previous attempts [48] generated face swapped images using an iOS app and a open-source face swap software using still images; though Zhou et al.”
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain (2021, methodology): “With all three kinds of compression setting, the proposed SPSL completely surpasses the orig-6 Method FF++ [37] Celeb-DF [26] Two-stream [49] 70.10 53.80 Meso4 [1] 84.70 54.80 MesoInception4 [1] 83.00 53.60 HeadPose [48] 47.30 54.60 FWA [25] 80.10 56.90 VA-MLP [29] 66.40 55.00 VA-LogReg 78.00 55.10…”
Multi-attentional Deepfake Detection (2021, background): “As the learning based methods become mainstream, some works [53, 34] have proposed frameworks which extract features from spatial domain and have achieved excellent performances on specific datasets.”
Celeb-DF: A New Dataset for DeepFake Forensics (2019, methodology): “Two-stream DNN-based forgery detection method [16], which is a general image forgery detection method that are used for DeepFake detection.”
Exposing DeepFake Videos By Detecting Face Warping Artifacts (2018, methodology): “[35] proposed two-stream CNN for face tampering detection.”