Opacus: User-Friendly Differential Privacy Library in PyTorch

By Ashkan Yousefpour, I. Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Gosh, Akash Bharadwaj, Jessica Zhao and 2 more. arXiv.org, 2021

551 citations recorded by the source corpus.

We introduce Opacus, a free, open-source PyTorch library for training deep learning models with differential privacy (hosted at opacus.ai). Opacus is designed for simplicity, flexibility, and speed. It provides a simple and user-friendly API, and enables machine learning practitioners to make a training pipeline private by adding as little as two lines to their code. It supports a wide variety of layers, including multi-head attention, convolution, LSTM, GRU (and generic RNN), and embedding, right out of the box and provides the means for supporting other user-defined layers. Opacus computes batched per-sample gradients, providing higher efficiency compared to the traditional"micro batch"approach. In this paper we present Opacus, detail the principles that drove its implementation and unique features, and benchmark it against other frameworks for training models with differential privacy as well as standard PyTorch.

Subfields: Artificial Intelligence, Computer Science Applications

View Paper: https://www.semanticscholar.org/paper/bea1187a1f8a68f1a93f0c2fa10d31f93a30f84e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

Citations in the corpus

On the Importance of Difficulty Calibration in Membership Inference Attacks (2021, methodology): “Differentially private training is done using the Opacus (Yousefpour et al., 2021) library.”
Individual Privacy Accounting via a Renyi Filter (2020, methodology): “We use the default architecture from the MNIST example of the PyTorch Opacus library [34].”
Large Language Models Can Be Strong Differentially Private Learners (2021, methodology): “Our JAX implementation is adapted from a codebase used in the work by Subramani et al. (2020) and utilizes the package flaxmodels for loading pretrained models.12 The Opacus (Yousefpour et al., 2021) baseline is based on version 0.14.0 of the library, where for a fair comparison, we also optimized t”
Analyzing Leakage of Personally Identifiable Information in Language Models (2023, methodology): “For DP training, we utilize the dp-transformers [61] library, which is a wrapper around Opacus [63].”
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy (2023, methodology): “PyTorch Opacus (Yousefpour et al., 2021) implements efficient per-example gradients for some types of neural network layers.”
TAN without a burn: Scaling Laws of DP-SGD (2022, methodology): “We use the Opacus (Yousefpour et al., 2021) library in Pytorch (Paszke et al.”