Embedding Watermarks into Deep Neural Networks

By Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, S. Satoh. International Conference on Multimedia Retrieval, 2017

768 citations recorded by the source corpus.

Significant progress has been made with deep neural networks recently. Sharing trained models of deep neural networks has been a very important in the rapid progress of research and development of these systems. At the same time, it is necessary to protect the rights to shared trained models. To this end, we propose to use digital watermarking technology to protect intellectual property and detect intellectual property infringement in the use of trained models. First, we formulate a new problem: embedding watermarks into deep neural networks. Second, we propose a general framework for embedding a watermark in model parameters, using a parameter regularizer. Our approach does not impair the performance of networks into which a watermark is placed because the watermark is embedded while training the host network. Finally, we perform comprehensive experiments to reveal the potential of watermarking deep neural networks as the basis of this new research effort. We show that our framework can embed a watermark during the training of a deep neural network from scratch, and during fine-tuning and distilling, without impairing its performance. The embedded watermark does not disappear even after fine-tuning or parameter pruning; the watermark remains complete even after 65% of parameters are pruned.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/69220b1c2bdfce552d97a3b142e8519a5d19e1a3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data (2020, background): “There are two categories of them: black-box trigger-set-based solutions [3, 54], and white-box feature-based solutions [43, 9, 41].”
Neural Trojans (2017, background): “[26] embedded watermarks into DNNs as a proof of authenticity.”
Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization (2021, background): “Song et al. (2017) and Uchida et al. (2017) inject a pattern that is similar to regular photograph water-marks (Cheng et al., 2021) into the least signiﬁcant bits of the model parameters and provide the corresponding decoding methods.”
HiDDeN: Hiding Data With Deep Networks (2018, positive): “[24] embed watermarks into the weights of trained neural networks; Fang et al.”
High Accuracy and High Fidelity Extraction of Neural Networks (2019, methodology): “Watermarking neural networks has been proposed [54, 55] to identify extracted models.”
High-Fidelity Extraction of Neural Network Models (2019, neutral): “Watermarking neural networks has been proposed [53, 54] to identify model extraction after-the-fact.”
Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints (2018, background): “It has been shown that neural networks can also be actively watermarked during training [65, 62].”
DeepSigns: An End-to-End Watermarking Framework for Ownership Protection of Deep Neural Networks (2019, methodology): “The works in [11, 12] encode the WM information in the weights of convolution layers, as opposed to the activation maps proposed by DeepSigns.”
Protecting Intellectual Property of Deep Neural Networks with Watermarking (2018, methodology): “[54] proposed the first method for embedding watermarks into deep neural networks.”
Hardware-Assisted Intellectual Property Protection of Deep Learning Models (2020, methodology): “In a white-box setting [23], the neural network architecture as well as the trained DL model parameters are made publicly available (e.g., Caffe’s Model Zoo and Amazon’s Alexa Skills) by a DL model owner [7], [19].”
DNN Intellectual Property Protection: Taxonomy, Methods, Attack Resistance, and Evaluations (2020, methodology): “In the verification stage, the watermark can be extracted from the weight of the marking layer [29].”
Survey of Attacks and Defenses on Edge-Deployed Neural Networks (2019, background): “Even if the party responsible for the theft attempts to prune or finetune the network, watermarks can be retained [62].”
A Systematic Review on Model Watermarking for Neural Networks (2020, background): “• Secrecy: Watermarking schemes that change the model parameters in a detectable way, as for example [Uchida et al., 2017], violate the secrecy requirement.”
Adversarial frontier stitching for remote neural network watermarking (2017, positive): “[35,22] published the first method for watermarking a neural network that might be publicly shared and thus for which traceability through ownership extraction is impor-”
DeepAttest: An End-to-End Attestation Framework for Deep Neural Networks (2019, methodology): “Compared to weight-based watermarking [48], DeepAttest’s FP extraction involves fewer computations since no extra sigmoid function is required.”
DAWN: Dynamic Adversarial Watermarking of Neural Networks (2019, methodology): “The first watermarking technique for DNNs [44] explicitly embeds additional information into the”
Dataset Inference: Ownership Resolution in Machine Learning (2021, background): “Since Uchida et al. (2017) embedded watermarks into neural networks and Adi et al. (2018) used them as signatures to claim possession, watermarks have been widely adopted as a way to resolve ownership claims.”
BlackMarks: Blackbox Multibit Watermarking for Deep Neural Networks (2018, methodology): “[6] takes the ﬁrst step towards DNN watermarking and embeds the WM in the weights of intermediate layers by training the model with an additional regularization loss.”
Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring (2018, methodology): “Moreover, the construction of [38] falls into the category of privately verifiable schemes, it can be made publicly verifiable using the same techniques as we show.”