A generic framework for privacy preserving deep learning

By T. Ryffel, Andrew Trask, Morten Dahl, Bobby Wagner, Jason V. Mancuso, Daniel Rueckert, Jonathan Passerat-Palmbach. 2018

495 citations recorded by the source corpus.

We detail a new framework for privacy preserving deep learning and discuss its assets. The framework puts a premium on ownership and secure processing of data and introduces a valuable representation based on chains of commands and tensors. This abstraction allows one to implement complex privacy preserving constructs such as Federated Learning, Secure Multiparty Computation, and Differential Privacy while still exposing a familiar deep learning API to the end-user. We report early results on the Boston Housing and Pima Indian Diabetes datasets. While the privacy features apart from Differential Privacy do not impact the prediction accuracy, the current implementation of the framework introduces a significant overhead in performance, which will be addressed at a later stage of the development. We believe this work is an important milestone introducing the first reliable, general framework for privacy preserving deep learning.

Subfields: Artificial Intelligence, Federated & Distributed Learning, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Federated & Distributed Learning

In Privacy & Security

Citations in the corpus

Advances and Open Problems in Federated Learning (2019, methodology): “These include TensorFlow Federated [38], Federated AI Technology Enabler [34], PySyft [342], Leaf [35], PaddleFL [36] and Clara Training Framework [33]; more details in Appendix A.”
Federated Learning With Differential Privacy: Algorithms and Performance Analysis (2019, background): “The work in [33] developed a chain abstraction model on tensors to efficiently override operations (or encode new ones) such as sending/sharing a tensor between workers, and then provided the elements to implement recently proposed DP and multiparty computation protocols using this framework.”
Flower: A Friendly Federated Learning Research Framework (2020, methodology): “Syft is able to communicate over the network, but only by connecting to data holding clients that act as servers themselves, which limits scalability.”
FedML: A Research Library and Benchmark for Federated Machine Learning (2020, methodology): “Therefore, when training large DNNs, we suggest using FedML ’s distributed computing paradigm, which is not supported by existing federated learning libraries like PySyft [28], LEAF [72], and TTF 10 .”
IBM Federated Learning: an Enterprise Framework White Paper V0.1 (2020, background): “A few approaches and libraries for FL address some of these issues, including [9], [28], and [4].”
Real-World Image Datasets for Federated Learning (2019, neutral): “Moreover, OpenMined proposes a python Library for secure, private deep learning [Ryffel et al., 2018].”
Self-supervised Cross-silo Federated Neural Architecture Search (2021, methodology): “open-sourced modeling tools, including TensorFlow Federated [4], PySyft [5], PaddleFL [6] and FATE [7] etc.”
Deep Anomaly Detection for Time-Series Data in Industrial IoT: A Communication-Efficient On-Device Federated Learning Approach (2020, methodology): “We implement the proposed framework by using Pytorch and PySyft [40].”
HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning (2019, methodology): “PySyft [34] employs the SPDZ protocol [12, 13].”
BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning (2020, methodology): “While we base our implementation on FATE, nothing pre-cludes it from being extended to the other frameworks such as TensorFlow Federated [20] and PySyft [49].”
Deep Learning for Cardiac Image Segmentation: A Review (2019, background): “Building a privacy-preserving algorithm requires to combine cryptography and deep learning together and to mix techniques from a wide range of subjects such as data analysis, distributed computing, federated learning, differential privacy, in order to achieve models with strong security, fast run ti”
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection (2019, background): “[145] and developed by OpenMined, is a python library that provides interfaces for developers to implement their training algorithm.”
Data Poisoning Attacks Against Federated Learning Systems (2020, background): “Companies and users alike are consequently faced with 2 fundamental questions in this reality of ML: (1) How can privacy concerns around such pervasive data collection be moderated without sacrificing the efficacy of ML models? and (2) How can ML models be trusted as accurate predictors? Federated M”
TiFL: A Tier-based Federated Learning System (2020, methodology): “ere has also been a rise of FL tools and framework development, such as Tensorow Federated [17], LEAF [7], PaddleFL [16] and PySy [30] to facilitate these demands.”
Federated Learning in Mobile Edge Networks: A Comprehensive Survey (2019, methodology): “• PySyft: PySyft [83] is a framework based on PyTorch for performing encrypted, privacy-preserving DL and implementations of related techniques, such as Secure Multiparty Computation (SMPC) and DP, in untrusted environments while protecting data.”
Industrial Artificial Intelligence in Industry 4.0 - Systematic Review, Challenges and Outlook (2020, background): “As such, it is imperative that future research addresses privacy-preserving constructs for AI, with some examples including secure multi-party computation schemes and differential privacy [97].”
FedBN: Federated Learning on Non-IID Features via Local Batch Normalization (2021, methodology): “…has much flexibility to be conbimed with other strategies, these merits allow us to easily integrate FedBN into existing tool-kits/systems, such as Pysyft (Ryffel et al., 2018), Google TFF (Google, 2020), Flower (Beutel et al., 2020), dlplatform (Kamp & Adilova, 2020) and FedML (He et al., 2020)8.”