P3SGD: Patient Privacy Preserving SGD for Regularizing Deep CNNs in Pathological Image Classification

By Bingzhe Wu, Shiwan Zhao, Guangyu Sun, Xiaolu Zhang, Zhong Su, Caihong Zeng, Zhihong Liu. 2019

45 citations recorded by the source corpus.

Recently, deep convolutional neural networks (CNNs) have achieved great success in pathological image classification. However, due to the limited number of labeled pathological images, there are still two challenges to be addressed: (1) overfitting: the performance of a CNN model is undermined by the overfitting due to its huge amounts of parameters and the insufficiency of labeled training data. (2) privacy leakage: the model trained using a conventional method may involuntarily reveal the private information of the patients in the training dataset. The smaller the dataset, the worse the privacy leakage. To tackle the above two challenges, we introduce a novel stochastic gradient descent (SGD) scheme, named patient privacy preserving SGD (P3SGD), which performs the model update of the SGD in the patient level via a large-step update built upon each patient's data. Specifically, to protect privacy and regularize the CNN model, we propose to inject the well-designed noise into the updates. Moreover, we equip our P3SGD with an elaborated strategy to adaptively control the scale of the injected noise. To validate the effectiveness of P3SGD, we perform extensive experiments on a real-world clinical dataset and quantitatively demonstrate the superior ability of P3SGD in reducing the risk of overfitting. We also provide a rigorous analysis of the privacy cost under differential privacy. Additionally, we find that the models trained with P3SGD are resistant to the model-inversion attack compared with those trained using non-private SGD.

Subfields: Artificial Intelligence, Biomedical Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

Citations in the corpus

AdaCliP: Adaptive Clipping for Private SGD (2019, background): “Recently, [37, 38] proposed adaptive strategies to select privacy parameters ′, δ′ for each iteration.”
A Study of Face Obfuscation in ImageNet (2021, background): “For example, model inversion attack recovers sensitive attributes (e.g., gender) of an individual given the model’s output (Fredrikson et al., 2014; 2015; Hamm, 2017; Li et al., 2019; Wu et al., 2019).”
The future of digital health with federated learning (2020, background): “However, the shared information may still indirectly expose private data used for local training, for example by model inversion [53] of the model updates, the gradients themselves [54] or adversarial attacks [55, 56].”
A Survey on Differential Privacy for Unstructured Data Content (2022, methodology): “Since we target applying DP directly to image content on users’ sides, we will not include the work [93, 110, 111], in which differential privacy is guaranteed in the Machine Learning training process.”